Structured output is enforced when the application requests it

AI models that are prompted to return JSON do not guarantee valid, correctly-shaped JSON on every response. Without validation via Zod, JSON Schema, or SDK-level structured output, a malformed or schema-violating AI response will either throw an unhandled exception at JSON.parse() or silently corrupt downstream logic — writing unexpected data to a database, crashing a rendering component, or causing a downstream API call to fail. OWASP LLM09 (Misinformation) covers the class of failures where AI output is consumed uncritically. CWE-20 (Improper Input Validation) applies directly: structured AI output is external input and must be treated as such.

Use SDK-level schema enforcement or Zod validation before consuming structured AI output. The preferred path is generateObject from the Vercel AI SDK:

const { object } = await generateObject({
  model: openai('gpt-4o'),
  schema: z.object({ summary: z.string(), tags: z.array(z.string()) }),
  prompt: userMessage
})

If using raw JSON.parse, always wrap in safeParse and handle the failure branch explicitly before writing to a database or passing to a renderer.

• ID: ai-response-quality.response-formatting.structured-output-compliance

• Severity: critical

• What to look for: Enumerate all relevant files and Identify any code paths where the application requests structured output from the AI (JSON mode, function calling, tool use, response_format: { type: "json_object" }, Zod schemas passed to the SDK, or structured output APIs). Check whether the application validates the returned structure before using it — look for try/catch around JSON.parse, Zod .parse() or .safeParse() calls, type guards, or schema validation on the parsed object. Check whether the application falls back gracefully if validation fails.

• Pass criteria: At least 1 implementation must be present. Every code path that requests structured AI output either (a) uses a schema-enforced SDK method (Vercel AI SDK generateObject, OpenAI structured outputs with a schema, Anthropic tool use) or (b) validates the parsed response with Zod, JSON Schema, or equivalent before using it in application logic. A fallback or error state exists for validation failures.

• Fail criteria: The application requests structured output (JSON mode or similar) but uses the result directly without validation — relying on the AI to always return valid, correctly-shaped JSON. A partial or incomplete implementation does not count as pass.

• Skip (N/A) when: No structured output requests are detected (no response_format, generateObject, tool calls, or explicit JSON-in-prompt patterns).

• Cross-reference: For data privacy assessment of AI response content, the AI Data Privacy audit covers PII detection and data retention policies.

• Detail on fail: "JSON.parse(response.content) called without validation or try/catch in api/generate/route.ts — malformed AI JSON will throw unhandled exception" (max 500 chars)

• Remediation: Always validate structured AI output before use:

  import { z } from 'zod'
  
  const ResponseSchema = z.object({
    summary: z.string(),
    tags: z.array(z.string()),
    confidence: z.number().min(0).max(1)
  })
  
  // Option A: Use SDK-level structured output (preferred)
  const { object } = await generateObject({
    model: openai('gpt-4o'),
    schema: ResponseSchema,
    prompt: '...'
  })
  
  // Option B: Validate manually
  const parsed = ResponseSchema.safeParse(JSON.parse(raw))
  if (!parsed.success) {
    return { error: 'Response validation failed' }
  }
  

  Never pass unvalidated AI-generated JSON into database writes, rendering logic, or downstream API calls.