# Tests run in CI

- **Pattern:** `ab-000305` (`ai-slop-test-theater.test-operability.tests-run-in-ci`)
- **Severity:** high
- **Lifecycle:** active
- **Last modified:** 2026-04-18
- **Canonical URL:** https://auditbuffet.com/patterns/ab-000305
- **License:** CC-BY-4.0 — attribute to AuditBuffet Pattern Catalog (https://auditbuffet.com/patterns/ab-000305)

## Why it matters

Tests that aren't wired into CI are effectively optional — they run on some developer machines, some of the time, with varying local environments. A PR that breaks a test can be merged because the author didn't run `npm test` locally and CI never did either. This is the most common failure mode for AI-generated test suites: the AI writes tests but doesn't update the CI workflow. SLSA Build L1 requires that the build process be scripted; ISO-25010:2011 maintainability requires that quality gates be automated and repeatable.

## Severity rationale

High because tests that don't run in CI provide no protection against regressions being merged to the main branch.

## Remediation

Add a test execution step to your CI workflow file. For GitHub Actions, a minimal addition to an existing `.github/workflows/ci.yml` is sufficient:

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm test
```

If no CI configuration exists at all, create `.github/workflows/ci.yml` with this content. The test step must appear in the same job that has access to the codebase.

## Detection

- **ID:** `tests-run-in-ci`
- **Severity:** `high`
- **What to look for:** Walk `.github/workflows/*.yml`, `.gitlab-ci.yml`, `.circleci/config.yml`, `bitbucket-pipelines.yml`, `azure-pipelines.yml`. For each CI config file, count all step entries that run tests: `npm test`, `npm run test`, `yarn test`, `pnpm test`, `bun test`, `vitest`, `jest`, `playwright test`, `cypress run`. If at least 1 CI config has at least 1 test execution step, tests run in CI.
- **Pass criteria:** At least 1 CI config file contains at least 1 test execution step. Report: "CI files inspected: X. Test execution steps found: Y."
- **Fail criteria:** Project has test files AND at least 1 CI config file AND no CI config runs tests.
- **Skip (N/A) when:** Project has 0 test files OR no CI configuration files exist.
- **Detail on fail:** `"Project has 47 test files and .github/workflows/ci.yml exists, but no CI step runs tests. Tests are written but never executed in CI — silently broken tests can be merged"`
- **Remediation:** Tests that don't run in CI are dead weight. Add a test step to your workflow:

  ```yaml
  # .github/workflows/ci.yml
  jobs:
    test:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v4
        - uses: actions/setup-node@v4
          with:
            node-version: '20'
        - run: npm ci
        - run: npm test
  ```

## External references

- iso-25010 maintainability.testability
- slsa build-l1

Taxons: operational-readiness

HTML version: https://auditbuffet.com/patterns/ab-000305