Uncertainty and confidence are signaled appropriately
Why it matters
A system prompt that instructs an AI to "always sound confident and authoritative" actively suppresses the model's natural uncertainty signaling — turning calibration failures into confident fabrications. Users in factual domains (legal, medical, financial, technical) rely on hedging language to know when to independently verify a claim. Stripping that signal is not a UX improvement; it is an epistemic hazard. OWASP LLM09 identifies overconfident AI output as a misinformation risk. NIST AI RMF MEASURE-2.5 requires that AI systems accurately represent their confidence levels to operators and users.
Severity rationale
High because suppressing uncertainty language causes users to act on fabricated or uncertain AI answers without the hedging signals that would prompt them to verify, directly enabling harm in factual-domain applications.
Remediation
Replace any "always sound confident" instruction with calibrated uncertainty guidance in src/lib/ai/system.ts or wherever your system prompt is defined:
const systemPrompt = `
Be direct when you are confident. When you are uncertain, say so:
"I believe...", "I'm not certain, but...", "You may want to verify this, but..."
Never present a guess as an established fact. If you don't know, say so plainly.
`
Do not overcorrect with blanket hedging on every statement — the goal is calibrated confidence, not universal uncertainty.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the AI Response Quality Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- owasp-llm:2025 · LLM09 — Misinformation
- nist-ai-rmf:1.0 · MEASURE 2.5 — AI system to be deployed meets stated trustworthiness criteria
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-response-quality·automated