User-controllable system prompt modification is prevented
Why it matters
Allowing users to modify the system prompt—whether through a "personality" slider, a custom persona field, or a request parameter—is architecturally equivalent to giving them write access to your application's security policy. OWASP LLM01:2025 and MITRE ATLAS AML.T0054 both identify this as a direct vector: the attacker simply submits their injection payload as a legitimate feature input rather than smuggling it through the user message. CWE-1427 captures this as uncontrolled influence over the inference contract. Personalization features that inject user text into the system message have caused production incidents where users granted themselves admin access, extracted other users' data, or caused the model to generate policy-violating content—all through the intended UI.
Severity rationale
Critical because user-controlled system prompt modification requires zero exploitation sophistication—the attacker uses the application's own feature to deliver the injection payload through a fully authorized request.
Remediation
Gate all system prompt selection through a server-side allowlist of pre-approved, developer-authored variants. No user-provided text enters the system message under any condition.
// src/lib/prompts.ts
const PROMPT_VARIANTS = {
assistant: ASSISTANT_SYSTEM_PROMPT,
analyst: ANALYST_SYSTEM_PROMPT,
support: SUPPORT_SYSTEM_PROMPT,
} as const
type PromptKey = keyof typeof PROMPT_VARIANTS
export function resolveSystemPrompt(key: unknown): string {
if (typeof key === 'string' && key in PROMPT_VARIANTS) {
return PROMPT_VARIANTS[key as PromptKey]
}
return PROMPT_VARIANTS.assistant // safe default
}
If your product requires user-customizable AI behavior, implement it as a named slot within a fixed template ("Respond in the user's preferred language: {language}"), where the slot value is validated and sanitized before insertion—never as free-text system prompt replacement.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the AI Prompt Injection Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- cwe · CWE-1427 — Improper Neutralization of Input Used in AI/ML Prompt Injection
- owasp-llm:2025 · LLM01 — Prompt Injection
- mitre-atlas:v4 · AML.T0054 — LLM Jailbreak
- nist-ai-rmf:1.0 · GOVERN 1.1 — Policies, processes, procedures, and practices across organization to manage AI risks
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-prompt-injection·automated