When server-side code passes raw database records — including user objects with hashed passwords, internal flags, or API credentials — into an AI prompt, that data becomes part of the inference context processed by an external third-party API. Even if the model does not echo the data verbatim, it may reference, paraphrase, or leak it in edge-case responses. GDPR Article 5(1)(c) (data minimization) requires that personal data is limited to what is necessary. CWE-200 (Exposure of Sensitive Information) and OWASP LLM06 (Sensitive Information Disclosure) both apply. Logging raw AI responses without access controls compounds the exposure by persisting the sensitive context to a log store.
Medium because exploitation requires either an unusual model behavior or log access, but the data minimization violation is present in every request and violates GDPR Article 5(1)(c) regardless of observed leakage.
Project only the fields needed for the task before passing data to the AI model:
// Before: full user object (includes hashed_password, internal_flags, api_key_hash)
const context = `User: ${JSON.stringify(user)}`
// After: explicit field selection
const context = `User display name: ${user.displayName}\nPlan: ${user.plan}`
Treat the AI API as an external third-party service — apply the same data minimization discipline you would before an external HTTP call. If logging AI responses for debugging, route them through a scrubber or store them behind access-controlled log infrastructure.
ID: ai-response-quality.hallucination-prevention.no-sensitive-data-leakage
Severity: medium
What to look for: Enumerate all relevant files and Examine what data is passed to the AI model in the context or user messages. Check whether server-side code passes database records, user profile data, API keys, internal configuration, or other sensitive data as part of the AI prompt without scrubbing. Check whether the system prompt or context injection includes raw database rows, full user objects, or environment variable values. Check whether AI responses are logged in raw form (without redaction) to a database, analytics service, or log provider.
Pass criteria: At least 1 implementation must be present. Context data passed to the AI is limited to what is necessary for the task. Sensitive fields (passwords, API keys, full PII beyond what's necessary) are excluded before prompt construction. AI response logs, if they exist, are treated as potentially sensitive data.
Fail criteria: Server-side code passes raw database objects or user records (including sensitive fields) into the prompt without filtering, or AI responses are logged without access controls to a logging service.
Skip (N/A) when: Application is a code assistant or local tool with no user data or database access.
Detail on fail: "Context injection in api/chat/route.ts passes full user object including hashed password and internal flags to AI prompt" (max 500 chars)
Remediation: Always project only the fields you intend to expose to the model:
// Before — full user object in prompt (risky)
const context = `User: ${JSON.stringify(user)}`
// After — only expose necessary fields
const context = `User: ${user.displayName}, Plan: ${user.plan}`
Treat the AI model as an external third-party service — it should receive only the minimum data needed to complete the task. For output safety and injection attacks, the AI Prompt Injection Audit covers how adversarial inputs can exfiltrate data passed in context.