# How AuditBuffet check IDs and scoring work

Canonical: https://auditbuffet.com/methodology · Last updated: 2026-11-12

AuditBuffet check IDs (ab-NNNNNN) are permanent identifiers for single, testable failure patterns in AI-built software. Each audit score is the severity-weighted share of applicable checks that passed.

## Check IDs

A check ID is `ab-` followed by at least six digits and nothing else (zero-padded to six, for example `ab-000183`). It is assigned once and never renamed.

An ab-ID is assigned once and never reused or renumbered. If a check is retired, its ID and page stay up, marked retired, so existing citations keep working. Gaps in the sequence are intentional and are never backfilled.

Dotted IDs (`<audit>.<section>.<check>`) and bare check slugs are aliases only. The ab-ID is primary in every citation, URL and structured-data field.

## Lifecycle

- **Active**: the check runs in at least one audit.
- **Deprecated**: the page still renders and names the check that replaces it (`deprecated_by`).
- **Retired**: the page stays up as a tombstone (HTTP 200, `noindex, follow`, never a 404 or redirect). Where a successor exists, the tombstone links to it.

## What is public and what is Pro

Every check's ID, title and description is public under CC-BY-4.0. Browse them all in the [pattern catalog](/patterns). The audit prompts and the detection logic for each check are part of Pro, except for the free Stack Scan.

## Scoring

Each check has a severity. Severities map to weights:

| Severity | Weight |
| --- | --- |
| critical | 10 |
| high | 3 |
| medium | 3 |
| low | 1 |
| info | 1 |

An audit score is the weight of the applicable checks that passed, divided by the weight of all applicable checks, times 100. Checks that are skipped (not applicable to your project) or that errored are left out of both sides. A score of 80 means the project passed checks carrying 80% of the applicable weight.

Scores map to letter grades:

| Grade | Label | Score |
| --- | --- | --- |
| A | Excellent | 90–100 |
| B | Good | 75–89 |
| C | Needs Work | 60–74 |
| D | Poor | 40–59 |
| F | Critical | 0–39 |

The raw pass, fail, skip and error results are the source of truth; scores and grades are derived from them.

## What "caught" means

We say a check **caught** an issue only after a run on our reproduction harness: a real source plus a before and after reproduction.

Otherwise we say "ab-NNNNNN is the class of issue this check looks for". If no check covers an issue, we say it is **not covered yet**. There is no partial verdict.

## Reproducibility

A reproducible result records four fields:

- **Audit name**: the audit that produced the score, for example `api-security`.
- **Audit version**: the audit's semantic version, printed in the prompt header.
- **Model**: the AI model that ran the audit.
- **Catalog commit**: the commit SHA of the check catalog the audit was built from.

The catalog is cited by `catalog.commit`. `catalog.version` is a display label only. The audit version is the number to match when you re-run.

How to reproduce:

1. Run `npx auditbuffet run <audit>` and check that the version in the prompt header matches the recorded audit version.
2. Open the project in your AI coding tool, using the recorded model (when shown).
3. Paste the prompt and let it run.
4. Submit the telemetry JSON it produces.

## Why tested audits

Every check is adversarially tested against clean and deliberately broken fixtures before it ships, then versioned and maintained as frameworks change. Running our tested audits costs far less than writing, testing and maintaining an equivalent check suite yourself. See [pricing](/pricing).
