AI response metadata stored for audit trail

NIST AI RMF GOVERN 1.7 requires organizations to maintain documentation of AI system decisions. ISO/IEC 27001:2022 A.8.15 (Logging) and NIST SP 800-53 Rev. 5 AU-3 require sufficient log detail to reconstruct events. When AI responses are stored without model version metadata, it becomes impossible to explain to a user why they received a particular response, identify which responses might be affected by a model deprecation or safety issue, or meet audit requests about AI decision provenance. Storing model: 'gpt-4o' alongside each response is a three-field schema change with significant operational value.

Extend your message schema to capture model version and provider alongside every AI-generated response. This costs one extra column and is populated directly from the API response object.

// schema.prisma
model Message {
  id           String   @id @default(cuid())
  role         String
  content      String
  model        String?  // e.g., "gpt-4o", "claude-3-5-sonnet-20241022"
  provider     String?  // e.g., "openai", "anthropic"
  promptTokens Int?
  outputTokens Int?
  createdAt    DateTime @default(now())
}

Populate it from the API response:

await db.message.create({
  data: {
    role: 'assistant',
    content: response.choices[0].message.content,
    model: response.model,          // comes back on every API response
    provider: 'openai',
    promptTokens: response.usage?.prompt_tokens,
    outputTokens: response.usage?.completion_tokens,
  }
})

The model field from the API response is always the resolved model name (e.g., gpt-4o-2024-11-20), which is more precise than the model alias you requested — store the API-returned value, not your request value.

• ID: ai-data-privacy.data-retention-deletion.ai-decision-audit-trail

• Severity: info

• What to look for: Enumerate every relevant item. Examine how AI responses are stored in the database. Look at the message/response schema for metadata fields beyond just the content: model or model_version field recording which AI model generated the response, created_at timestamp, provider identifier, and optionally token usage. Also look for logging of model version alongside stored responses in the AI handler code.

• Pass criteria: At least 1 of the following conditions is met. Stored AI responses include at minimum a timestamp and the model identifier (e.g., gpt-4o, claude-3-5-sonnet-20241022) — enough to reconstruct which model version produced each response.

• Fail criteria: AI responses are stored as plain text content only, with no associated model version, provider, or timestamp metadata.

• Skip (N/A) when: AI responses are not persisted — the application uses only streaming responses that are not stored anywhere.

• Detail on fail: "AI responses stored in [table] contain only content — no model version, provider, or generation metadata recorded"

• Remediation: Knowing which model version produced a given AI response is valuable for debugging unexpected outputs, explaining AI decisions to users if challenged, and identifying responses that might need review after a model update.

  Extend your message schema:

  model Message {
    id           String   @id @default(cuid())
    role         String   // "user" | "assistant"
    content      String   // Consider encrypting at rest
    // Metadata for assistant messages:
    model        String?  // e.g., "gpt-4o", "claude-3-5-sonnet-20241022"
    provider     String?  // e.g., "openai", "anthropic"
    promptTokens Int?
    outputTokens Int?
    createdAt    DateTime @default(now())
  }
  

  Populate it from the API response:

  await db.message.create({
    data: {
      role: 'assistant',
      content: response.choices[0].message.content,
      model: response.model,
      provider: 'openai',
      promptTokens: response.usage?.prompt_tokens,
      outputTokens: response.usage?.completion_tokens,
    }
  })