Tool and function call arguments are validated before execution

When a model is given tools (function calling, tool use), a successful prompt injection no longer just changes the model's words—it causes the model to call real functions with attacker-specified arguments. OWASP LLM06:2025 (Excessive Agency) directly addresses this: an AI agent that can take consequential actions without independent authorization checks is a force-multiplier for any injection. MITRE ATLAS AML.T0051 and CWE-1427 both apply to the unvalidated dispatch of tool calls. A tool that performs database writes, sends emails, or makes API calls on the model's instruction—without re-verifying authorization at the execution layer—can be triggered by an injected instruction to perform actions the authenticated user never authorized. NIST AI RMF GOVERN 1.1 requires clear boundaries around what an AI system is authorized to do.

Validate tool calls at three layers: allowlist the tool name, schema-validate the arguments, and enforce authorization inside the tool function independent of the model's request.

// src/lib/tools.ts
const ALLOWED_TOOLS = ['search_docs', 'get_user_profile', 'create_draft'] as const
type AllowedTool = (typeof ALLOWED_TOOLS)[number]

for (const toolCall of completion.choices[0]?.message?.tool_calls ?? []) {
  // Layer 1: allowlist check
  if (!(ALLOWED_TOOLS as readonly string[]).includes(toolCall.function.name)) {
    throw new Error(`Blocked unknown tool: ${toolCall.function.name}`)
  }

  // Layer 2: argument schema validation
  const schema = ToolSchemas[toolCall.function.name as AllowedTool]
  const args = schema.parse(JSON.parse(toolCall.function.arguments))

  // Layer 3: authorization inside the tool function (never trusts caller intent)
  await TOOL_HANDLERS[toolCall.function.name as AllowedTool](args, session)
}

The authorization check inside each tool handler is non-negotiable: it must verify the acting user has permission for the specific operation, not just that the model requested it.