AI-generated content is moderated — no deepfakes, disinformation, or unfiltered harmful output

Both Apple's Generative AI guidelines (2024) and Google Play's AI-generated content policy require that AI output be moderated before display, and that synthetic media of real people carry explicit consent and disclosure. OWASP LLM02 (Insecure Output Handling) and NIST AI RMF GOVERN-1.7 classify direct pass-through of LLM output to users as a governance failure. Practical consequences: a deepfake face-swap app without age verification will be rejected on first review; an AI chat that produces unmoderated harmful content will be removed after a user report regardless of the developer's intent.

Add a content moderation call between AI output and display for all text-generation features:

// src/lib/ai.ts
async function generateAndScreen(prompt: string): Promise<string> {
  const response = await openai.chat.completions.create({ ... });
  const content = response.choices[0].message.content ?? '';

  const mod = await openai.moderations.create({ input: content });
  if (mod.results[0].flagged) {
    return "I can't help with that.";
  }
  return content;
}

Label all AI-generated content with a visible badge, especially in health, news, legal, or financial contexts. Lock the system prompt — never expose it to user configuration via a settings screen. For synthetic media features, add an explicit consent flow before enabling face-swap or voice cloning, and watermark all output.