A .skip, xit, or xdescribe without a justifying comment is a permanently-disabled test. It stops failing in CI, the broken behavior it was covering goes undetected, and because nobody knows why it was skipped, nobody knows it's safe to re-enable. AI-generated test suites often skip tests for code paths that haven't been implemented yet and never re-enable them. Over time these accumulate into a graveyard of skipped tests that silently shrink effective test coverage. ISO-25010:2011 maintainability requires that disabled tests be traceable to a documented reason.
High because unjustified skips reduce effective coverage silently — the skipped test no longer fails in CI even if the production code it covered is broken.
For every .skip call that has no comment above it and no justifying text in its name, either fix the underlying issue and re-enable it, or add a comment with a tracking issue and a target date. Never leave a skip anonymous:
// Before: permanently skipped, unknown reason
it.skip('creates a user', async () => { ... })
// After option A: fix and re-enable
it('creates a user', async () => {
const user = await createUser({ email: 'a@b.com' })
expect(user.id).toBeDefined()
})
// After option B: document the skip explicitly
// TODO(ISSUE-423): blocked on user service migration, re-enable by 2026-Q2
it.skip('creates a user (blocked on migration)', async () => { ... })
ID: ai-slop-test-theater.coverage-reality.no-skip-without-justification
Severity: high
What to look for: Walk all test files. Count all occurrences of it.skip(, test.skip(, describe.skip(, xit(, xdescribe(, xtest(. For each, verify either: (a) the line above contains a comment explaining why, OR (b) the call's first-argument string contains a justification (e.g., it.skip('wait until issue #42 is fixed')).
Pass criteria: 100% of .skip calls have a justification (comment or named string). Report: "X skipped tests inspected, Y justified, 0 unjustified."
Fail criteria: At least 1 .skip call has no comment and no descriptive name.
Skip (N/A) when: No .skip/xit/xdescribe/xtest calls in any test file.
Detail on fail: "2 unjustified skips: tests/auth.test.ts line 34 'it.skip(\"creates a user\", ...)' (no comment, no reason), tests/api.test.ts line 12 'describe.skip(\"webhook handler\", ...)'"
Remediation: A .skip without justification is a permanent disabled test. Either fix the test or document why it's disabled:
// Bad: silently disabled forever
it.skip('creates a user', async () => { ... })
// Good: documented skip with a deadline
// TODO(ISSUE-123): re-enable after we migrate the user service in Q2
it.skip('creates a user (blocked on migration)', async () => { ... })