Test Reality Audit

ai-slop-test-theater · v1.0.1 · paid

Catches AI-generated test suites that look impressive but don't actually test anything — assertion-free test files, mock-saturated tests that test the mocks, tautologies, skipped tests in CI, and missing E2E coverage on critical user flows.

Assertion Quality

assertion-quality · weight 0.3

Every test file contains at least one assertion
ab-000291
critical
No tautological assertions
ab-000292
critical
No catch-all try/catch blocks that swallow errors
ab-000293
high
Assertion density is reasonable per test
ab-000294
medium
Test names are descriptive
ab-000295
low

Mock & Setup Hygiene

mock-hygiene · weight 0.25

Mocks do not swallow the system under test
ab-000296
high
Setup and teardown are balanced
ab-000297
medium
Fake timers are paired with explicit clock control
ab-000298
low
Mock libraries are imported only in test files
ab-000299
medium

Coverage Reality

coverage-reality · weight 0.25

Critical source files have at least one corresponding test
ab-000300
critical
E2E tests exist for the primary user flow
ab-000301
high
No `.skip` patterns in shipping tests without justification
ab-000302
high
Test bodies are not empty or TODO-only
ab-000303
medium
Database/network tests are isolated from production data
ab-000304
low

Test Operability

test-operability · weight 0.2

Tests run in CI
ab-000305
high
Coverage configuration is present
ab-000306
medium
Tests use a fake clock when time is involved
ab-000307
info
Test database is separate from dev database
ab-000308
info