System prompt includes PII safety instructions

OWASP LLM01 (Prompt Injection) attackers often use injected instructions to extract or reproduce PII that has been provided in the conversation context. Adding behavioral guardrails to the system prompt — "do not repeat personal information the user shares" — reduces the surface area for accidental PII regurgitation. NIST AI RMF MANAGE 2.2 calls for implementing controls to minimize AI-generated harm. While system prompt instructions are not a technical security control and can be overridden by adversarial prompts, they meaningfully reduce accidental PII generation from normal interactions and demonstrate a documented intent to protect user data.

Add explicit PII safety instructions to your system prompt constant in lib/ai/ or wherever your system prompt is defined.

// lib/ai/system-prompt.ts
export const SYSTEM_PROMPT = `You are a helpful assistant for [App Name].

Privacy guidelines:
- Do not ask users for personal information (email, phone, address, SSN, etc.)
- If a user shares personal information, do not repeat it back or include it in your response
- If asked to generate personal information about real individuals, decline politely
- Focus responses on the task without referencing identifying details the user may have shared`

Store the system prompt as a versioned constant so changes are tracked in version control. For higher-risk applications, add domain-specific prohibitions (e.g., "Do not provide medical diagnoses" or "Do not discuss specific account balances").