AI Response Quality Audit

ai-response-quality · v1.1.0 · paid

Quality and trustworthiness assessment of AI-generated responses, including output formatting, context grounding, and communication of uncertainty or knowledge gaps.

Response Formatting

response-formatting · weight 0.25

Markdown rendering is enabled for AI response surfaces
ab-000197
critical
Structured output is enforced when the application requests it
ab-000198
critical
Code blocks are used in AI responses when code is present
ab-000199
high
Response length is proportional to query complexity
ab-000200
medium
Truncated responses are detected and handled gracefully
ab-000201
low

Source Attribution

source-attribution · weight 0.25

No fabricated references or citations
ab-000202
critical
External knowledge claims include appropriate attribution cues
ab-000203
high
Knowledge cutoff date is disclosed where relevant
ab-000204
high
RAG retrieval sources are passed through to the model context
ab-000205
medium
Attribution format is consistent across response types
ab-000206
low

Hallucination Prevention

hallucination-prevention · weight 0.3

Responses are grounded in provided context (not confabulated)
ab-000207
high
Uncertainty and confidence are signaled appropriately
ab-000208
high
Out-of-scope requests are refused gracefully
ab-000209
medium
Sensitive data is not leaked in AI responses
ab-000210
medium
Factual claims are bounded to deterministic scope
ab-000211
medium
AI explicitly acknowledges knowledge gaps rather than guessing
ab-000212
info

Response Management

response-management · weight 0.2

Content safety filtering is active on AI responses
ab-000213
high
Responses are consistent for semantically equivalent queries
ab-000214
low
Context window utilization is monitored or bounded
ab-000215
low
Response metadata is available for debugging and observability
ab-000216
info