Tests run in CI

Tests that aren't wired into CI are effectively optional — they run on some developer machines, some of the time, with varying local environments. A PR that breaks a test can be merged because the author didn't run npm test locally and CI never did either. This is the most common failure mode for AI-generated test suites: the AI writes tests but doesn't update the CI workflow. SLSA Build L1 requires that the build process be scripted; ISO-25010:2011 maintainability requires that quality gates be automated and repeatable.

Add a test execution step to your CI workflow file. For GitHub Actions, a minimal addition to an existing .github/workflows/ci.yml is sufficient:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm test

If no CI configuration exists at all, create .github/workflows/ci.yml with this content. The test step must appear in the same job that has access to the codebase.

• ID: tests-run-in-ci

• Severity: high

• What to look for: Walk .github/workflows/*.yml, .gitlab-ci.yml, .circleci/config.yml, bitbucket-pipelines.yml, azure-pipelines.yml. For each CI config file, count all step entries that run tests: npm test, npm run test, yarn test, pnpm test, bun test, vitest, jest, playwright test, cypress run. If at least 1 CI config has at least 1 test execution step, tests run in CI.

• Pass criteria: At least 1 CI config file contains at least 1 test execution step. Report: "CI files inspected: X. Test execution steps found: Y."

• Fail criteria: Project has test files AND at least 1 CI config file AND no CI config runs tests.

• Skip (N/A) when: Project has 0 test files OR no CI configuration files exist.

• Detail on fail: "Project has 47 test files and .github/workflows/ci.yml exists, but no CI step runs tests. Tests are written but never executed in CI — silently broken tests can be merged"

• Remediation: Tests that don't run in CI are dead weight. Add a test step to your workflow:

  # .github/workflows/ci.yml
  jobs:
    test:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v4
        - uses: actions/setup-node@v4
          with:
            node-version: '20'
        - run: npm ci
        - run: npm test