Skip to main content

Database/network tests are isolated from production data

ab-000304 · ai-slop-test-theater.coverage-reality.database-or-network-tests-isolated
Severity: lowactive

Why it matters

Tests that write to a database without a separate test database URL will insert, update, and delete rows in the development database. Over time this pollutes developer workflows with synthetic data, breaks count-based assertions as rows accumulate, and creates the risk that a destructive test (deleteMany, truncate) runs against real data if configuration is accidentally shared across environments. ISO-25010:2011 recoverability requires that test execution be isolated from other system states.

Severity rationale

Low because the impact is typically pollution and flakiness rather than immediate data loss, unless a destructive query is involved.

Remediation

Create a .env.test file with a separate DATABASE_URL pointing to a dedicated test database. Reference it explicitly in vitest.config.ts or your test runner configuration so it's impossible to accidentally run tests against the dev database:

# .env.test
DATABASE_URL=postgresql://user:pass@localhost:5432/myapp_test
// vitest.config.ts
export default defineConfig({
  test: {
    env: { DATABASE_URL: process.env.TEST_DATABASE_URL ?? '' },
    setupFiles: ['./tests/setup.ts'],
  },
})

Detection

  • ID: ai-slop-test-theater.coverage-reality.database-or-network-tests-isolated

  • Severity: low

  • What to look for: When at least 1 test file uses a database (prisma.X.create(/db.insert(/Model.create( calls), verify the project has a separate test database configuration: a .env.test file, OR a test-specific DATABASE_URL reference in vitest.config.ts/jest.config.ts/playwright.config.ts, OR a package.json test script that sets DATABASE_URL=.... Count all test files using DB calls and check whether the test database is isolated from the dev database (different URL/file path).

  • Pass criteria: When tests touch the database, the project has a separate test DB configuration. Report: "X test files use the database, test DB isolated: [yes/no]."

  • Fail criteria: Tests use the database but no separate test DB config exists.

  • Skip (N/A) when: No test files contain database calls.

  • Detail on fail: "5 test files use the database but no .env.test, no test-specific DATABASE_URL in vitest.config.ts. Tests likely run against the dev database, polluting it"

  • Remediation: Tests against the dev database pollute it and create flakiness. Use a separate test database:

    # .env.test
    DATABASE_URL=postgresql://user:pw@localhost:5432/myapp_test
    
    // vitest.config.ts
    export default defineConfig({
      test: {
        env: {
          DATABASE_URL: process.env.TEST_DATABASE_URL,
        },
      },
    })
    

External references

Taxons

History