Mocks do not swallow the system under test
Why it matters
When a test mocks every export from the file it is supposed to test, the test exercises the mock — not the production code. The function under test is never called. The test will pass even if the real implementation is completely deleted. This pattern appears in AI-generated tests where the model confuses "mock the dependencies of X" with "mock X itself." The test suite accumulates green results that have zero correlation with production behavior, creating false confidence. ISO-25010:2011 testability requires that tests exercise the actual system under test.
Severity rationale
High because mock-saturated tests provide a false signal of correctness — the production code could be entirely broken while every test passes.
Remediation
Mock external dependencies (databases, APIs, third-party SDKs), never the function being tested. The unit under test must be imported and called directly:
// Before: tests the mock, not the real calculateTax
import { calculateTax } from './tax'
vi.mock('./tax', () => ({ calculateTax: vi.fn(() => 100) }))
it('calculates tax', () => {
expect(calculateTax(1000)).toBe(100) // always true
})
// After: real function runs, only its dependency is mocked
import { calculateTax } from './tax'
import { taxRateApi } from './tax-rate-api'
vi.mock('./tax-rate-api', () => ({ taxRateApi: { fetch: vi.fn(() => 0.1) } }))
it('calculates tax at 10% rate', async () => {
const tax = await calculateTax(1000)
expect(tax).toBe(100)
})
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the Test Reality Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- iso-25010:2011 · maintainability.testability — Testability (maintainability sub-characteristic)
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-slop-test-theater·automated