Content moderation is applied to user inputs

Input content moderation is the cheapest effective first-line filter against harmful and adversarial content reaching your model. Without it, your AI feature accepts arbitrary text from any user—including coordinated abuse, prompt injection probes, and policy-violating requests. OWASP LLM01:2025 identifies moderation as a required control; NIST AI RMF MANAGE 1.3 requires active management of harm vectors in deployed AI systems. For consumer-facing applications, unmoderated inputs expose the platform to both regulatory risk and reputation damage when the model produces harmful outputs in response to adversarial prompts that a moderation layer would have blocked at the door.

Call the OpenAI Moderation endpoint before your primary completion call. It is free, averages under 100ms, and blocks a broad category of harmful and adversarial content.

// src/app/api/chat/route.ts
export async function POST(req: Request) {
  const { message } = await ChatInputSchema.parse(await req.json())

  // Moderation gate — runs before the primary completion
  const mod = await openai.moderations.create({ input: message })
  if (mod.results[0]?.flagged) {
    return Response.json({ error: 'Message not allowed' }, { status: 400 })
  }

  // Proceed with primary completion only if moderation passes
  const completion = await openai.chat.completions.create({ ... })
}

If you use a provider other than OpenAI, Perspective API (Google) covers similar categories and has a free tier for low-volume applications. Log the moderation result (not the full message content) for monitoring.