Structured output is validated before use
Why it matters
Models instructed to return JSON do not always comply—and even compliant responses may return unexpected field values, missing required properties, or subtly wrong data types. CWE-20 (Improper Input Validation) applies when structured AI output is consumed without validation. OWASP LLM02:2025 identifies unvalidated structured output as a direct path to application logic corruption. When action decisions ("delete this record," "charge this card," "send this email") are derived from parsed model output without schema validation, a malformed response—whether from a model error or an adversarially-triggered injection—can trigger unintended operations. NIST AI RMF MEASURE 2.6 requires demonstrable controls on AI system outputs before they trigger downstream effects.
Severity rationale
High because structured output drives application actions; a malformed or adversarially-crafted response accepted without validation can trigger unintended data mutations, charges, or communications.
Remediation
Validate all structured AI output against a typed Zod schema using .safeParse() before accessing any field. Never trust that the model returned the expected shape.
// Zod schema matching your expected model output
const AIResponseSchema = z.object({
action: z.enum(['create', 'update', 'delete']),
targetId: z.string().uuid(),
reason: z.string().max(200)
})
const raw = completion.choices[0]?.message?.content ?? ''
let jsonParsed: unknown
try {
jsonParsed = JSON.parse(raw)
} catch {
return Response.json({ error: 'Model returned invalid JSON' }, { status: 500 })
}
const result = AIResponseSchema.safeParse(jsonParsed)
if (!result.success) {
console.error('Unexpected AI output:', result.error)
return Response.json({ error: 'Could not process AI response' }, { status: 500 })
}
// result.data is typed and validated — safe to act on
const { action, targetId } = result.data
For function calling and tool use, apply the same .safeParse() to the arguments field before passing values to your tool handler.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the AI Prompt Injection Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- cwe · CWE-20 — Improper Input Validation
- owasp-llm:2025 · LLM02 — Insecure Output Handling
- nist-ai-rmf:1.0 · MEASURE 2.6 — Evaluate AI system trustworthiness and risk posture
Taxons
History
- 2026-04-18·v1.0.0·Initial import from ai-prompt-injection·automated