Content safety filtering is active on AI responses

Without a content safety layer, every user-visible AI response is raw model output — including responses to adversarially crafted inputs designed to elicit harmful content. OWASP LLM05 (Improper Output Handling) covers this failure mode. NIST AI RMF MAP-5.1 requires that AI systems have mapped and mitigated output harms before deployment to users. A single unfiltered harmful response to a manipulated prompt is sufficient for reputational, legal, and regulatory exposure — particularly for applications accessible to minors or vulnerable populations. Provider-level safety settings are necessary but insufficient without application-layer verification.

Add at least one content safety mechanism to the response pipeline in src/app/api/chat/route.ts:

// Option A: OpenAI Moderation API (add after generating the response)
const moderation = await openai.moderations.create({ input: aiResponse.content })
if (moderation.results[0]?.flagged) {
  return NextResponse.json({ error: 'Response flagged by safety filter' }, { status: 422 })
}

// Option B: Anthropic Claude — configure system prompt safety framing
// Option C: Azure AI Content Safety or AWS Bedrock Guardrails for provider-level filtering

For adversarial inputs that attempt to bypass safety filters via prompt injection, the AI Prompt Injection Audit covers jailbreak and injection vectors.