AuditBuffet check IDs (ab-NNNNNN) are permanent identifiers for single, testable failure patterns in AI-built software. Each audit score is the severity-weighted share of applicable checks that passed.
A check ID is ab- followed by at least six digits and nothing else (zero-padded to six, for example ab-000183). It is assigned once and never renamed.
An ab-ID is assigned once and never reused or renumbered. If a check is retired, its ID and page stay up, marked retired, so existing citations keep working. Gaps in the sequence are intentional and are never backfilled.
Dotted IDs (<audit>.<section>.<check>) and bare check slugs are aliases only. The ab-ID is primary in every citation, URL and structured-data field.
deprecated_by).noindex, follow, never a 404 or redirect). Where a successor exists, the tombstone links to it.Every check's ID, title and description is public under CC-BY-4.0. Browse them all in the pattern catalog. The audit prompts and the detection logic for each check are part of Pro, except for the free Stack Scan.
Each check has a severity. Severities map to weights:
| Severity | Weight |
|---|---|
| critical | 10 |
| high | 3 |
| medium | 3 |
| low | 1 |
| info | 1 |
An audit score is the weight of the applicable checks that passed, divided by the weight of all applicable checks, times 100. Checks that are skipped (not applicable to your project) or that errored are left out of both sides. A score of 80 means the project passed checks carrying 80% of the applicable weight.
Scores map to letter grades:
| Grade | Label | Score |
|---|---|---|
| A | Excellent | 90–100 |
| B | Good | 75–89 |
| C | Needs Work | 60–74 |
| D | Poor | 40–59 |
| F | Critical | 0–39 |
The raw pass, fail, skip and error results are the source of truth; scores and grades are derived from them.
We say a check caught an issue only after a run on our reproduction harness: a real source plus a before and after reproduction.
Otherwise we say "ab-NNNNNN is the class of issue this check looks for". If no check covers an issue, we say it is not covered yet. There is no partial verdict.
A reproducible result records four fields:
api-security.The catalog is cited by catalog.commit. catalog.version is a display label only. The audit version is the number to match when you re-run.
How to reproduce:
npx auditbuffet run <audit> and check that the version in the prompt header matches the recorded audit version.Every check is adversarially tested against clean and deliberately broken fixtures before it ships, then versioned and maintained as frameworks change. Running our tested audits costs far less than writing, testing and maintaining an equivalent check suite yourself. See pricing.