Tests run in CI
Why it matters
Tests that aren't wired into CI are effectively optional — they run on some developer machines, some of the time, with varying local environments. A PR that breaks a test can be merged because the author didn't run npm test locally and CI never did either. This is the most common failure mode for AI-generated test suites: the AI writes tests but doesn't update the CI workflow. SLSA Build L1 requires that the build process be scripted; ISO-25010:2011 maintainability requires that quality gates be automated and repeatable.
Severity rationale
High because tests that don't run in CI provide no protection against regressions being merged to the main branch.
Remediation
Add a test execution step to your CI workflow file. For GitHub Actions, a minimal addition to an existing .github/workflows/ci.yml is sufficient:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- run: npm ci
- run: npm test
If no CI configuration exists at all, create .github/workflows/ci.yml with this content. The test step must appear in the same job that has access to the codebase.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the Test Reality Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- iso-25010:2011 · maintainability.testability — Testability — tests executed automatically in CI pipeline
- slsa:1.0 · build-l1 — SLSA Build Level 1 — scripted, verifiable build process
- soc2:2017 · CC8.1 — Change management
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-slop-test-theater·automated
- 2026-07-06·v1.0.1·Add SOC 2 TSC mapping (CC8.1) from the verified compliance crosswalk register (docs/compliance/).·by soc2-mapping-enrichment-ship-2