No `.skip` patterns in shipping tests without justification
Why it matters
A .skip, xit, or xdescribe without a justifying comment is a permanently-disabled test. It stops failing in CI, the broken behavior it was covering goes undetected, and because nobody knows why it was skipped, nobody knows it's safe to re-enable. AI-generated test suites often skip tests for code paths that haven't been implemented yet and never re-enable them. Over time these accumulate into a graveyard of skipped tests that silently shrink effective test coverage. ISO-25010:2011 maintainability requires that disabled tests be traceable to a documented reason.
Severity rationale
High because unjustified skips reduce effective coverage silently — the skipped test no longer fails in CI even if the production code it covered is broken.
Remediation
For every .skip call that has no comment above it and no justifying text in its name, either fix the underlying issue and re-enable it, or add a comment with a tracking issue and a target date. Never leave a skip anonymous:
// Before: permanently skipped, unknown reason
it.skip('creates a user', async () => { ... })
// After option A: fix and re-enable
it('creates a user', async () => {
const user = await createUser({ email: 'a@b.com' })
expect(user.id).toBeDefined()
})
// After option B: document the skip explicitly
// TODO(ISSUE-423): blocked on user service migration, re-enable by 2026-Q2
it.skip('creates a user (blocked on migration)', async () => { ... })
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the Test Reality Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- iso-25010:2011 · maintainability.modifiability — Maintainability (modifiability sub-characteristic)
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-slop-test-theater·automated