System prompt or AI persona is disclosed or customizable
Why it matters
Hidden system prompts that shape AI behavior without user knowledge undermine informed consent and erode trust when discovered. NIST AI RMF 1.0 GOVERN-1.1 requires that AI system transparency be established as organizational policy — this extends to deployed products. A concealed prompt that restricts topics, imposes a persona, or steers conclusions while the UI presents the product as neutral is a form of deceptive design. Users who later discover undisclosed behavioral constraints lose trust in the product entirely, and in regulated contexts (finance, healthcare, legal) hidden constraints create liability.
Severity rationale
Medium because the harm is reputational and trust-based rather than immediate data exposure, but discovery of a hidden behavioral prompt causes disproportionate user backlash.
Remediation
Add a visible disclosure wherever users interact with the AI. This does not require exposing the full system prompt text — a plain-language summary of what the AI is configured to do and not do is sufficient to satisfy NIST AI RMF GOVERN-1.1.
// In src/components/chat/AiInfoPanel.tsx or a settings modal
<div className="rounded-lg bg-muted p-4 text-sm space-y-1">
<p className="font-medium">How this AI is configured</p>
<p className="text-muted-foreground">
This assistant is focused on {domain} and will not {restrictions}.
{allowsCustomization && 'You can adjust its behavior in Settings → AI Persona.'}
</p>
</div>
For products where customization is a selling point, expose a system prompt textarea in src/app/settings/page.tsx under an advanced section.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the AI UX Patterns Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- nist-ai-rmf:1.0 · GOVERN 1.1 — Policies for transparency and accountability in AI system configuration
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-ux-patterns·automated