Structured output is validated before use

Models instructed to return JSON do not always comply—and even compliant responses may return unexpected field values, missing required properties, or subtly wrong data types. CWE-20 (Improper Input Validation) applies when structured AI output is consumed without validation. OWASP LLM02:2025 identifies unvalidated structured output as a direct path to application logic corruption. When action decisions ("delete this record," "charge this card," "send this email") are derived from parsed model output without schema validation, a malformed response—whether from a model error or an adversarially-triggered injection—can trigger unintended operations. NIST AI RMF MEASURE 2.6 requires demonstrable controls on AI system outputs before they trigger downstream effects.

Validate all structured AI output against a typed Zod schema using .safeParse() before accessing any field. Never trust that the model returned the expected shape.

// Zod schema matching your expected model output
const AIResponseSchema = z.object({
  action: z.enum(['create', 'update', 'delete']),
  targetId: z.string().uuid(),
  reason: z.string().max(200)
})

const raw = completion.choices[0]?.message?.content ?? ''

let jsonParsed: unknown
try {
  jsonParsed = JSON.parse(raw)
} catch {
  return Response.json({ error: 'Model returned invalid JSON' }, { status: 500 })
}

const result = AIResponseSchema.safeParse(jsonParsed)
if (!result.success) {
  console.error('Unexpected AI output:', result.error)
  return Response.json({ error: 'Could not process AI response' }, { status: 500 })
}

// result.data is typed and validated — safe to act on
const { action, targetId } = result.data

For function calling and tool use, apply the same .safeParse() to the arguments field before passing values to your tool handler.

• ID: ai-prompt-injection.output-filtering.structured-output-validation

• Severity: high

• What to look for: Count all endpoints that parse LLM output as structured data (JSON, XML, function calls). For each, if the application requests structured output from the model (JSON mode, function calling, tool use, or a prompt that asks for JSON), check whether the response is parsed and validated against a schema before the data is used. Look for Zod, Yup, Joi, or JSON Schema validation on parsed AI output. Also check whether JSON parsing is done with error handling or whether malformed output would throw an unhandled exception.

• Pass criteria: Structured AI output is: (1) parsed with error handling, and (2) validated against a typed schema before the data is used. Zod .safeParse() or equivalent used with the failure case handled — 100% of structured LLM outputs must be validated against a schema before use. Report: "X structured output parsers found, all Y validate with schemas."

• Fail criteria: AI output is parsed with JSON.parse() without error handling, or parsed output is used directly without schema validation (trusting that the model returned the expected shape).

• Skip (N/A) when: No AI provider integration detected, or the application uses the model output only as plain text and never parses it as structured data.

• Detail on fail: "lib/ai.ts parses the AI response with JSON.parse() inside a try/catch but does not validate the resulting object against a schema before using its properties" or "Function call output is used directly without validating required fields are present"

• Remediation: Models can return malformed, unexpected, or adversarially-crafted JSON that breaks your application logic if used without validation:

  const ResponseSchema = z.object({
    action: z.enum(['create', 'update', 'delete']),
    targetId: z.string().uuid(),
    reason: z.string().max(200)
  })
  
  const raw = completion.choices[0]?.message?.content ?? ''
  const parsed = ResponseSchema.safeParse(JSON.parse(raw))
  
  if (!parsed.success) {
    // Handle validation failure — do not proceed with the action
    console.error('Unexpected AI output shape:', parsed.error)
    return Response.json({ error: 'Could not process AI response' }, { status: 500 })
  }
  
  const { action, targetId, reason } = parsed.data // typed and validated
  

  For tool/function call output, apply the same validation to the arguments field before acting on it.