Context window utilization is tracked and surfaced

Without context window tracking, your application flies blind into the model's hard token ceiling. When a conversation grows long enough to exceed that limit, the provider returns HTTP 400 context_length_exceeded — a hard error that crashes the session with no warning, no fallback, and no opportunity for graceful recovery. OWASP LLM10 and CWE-770 (Allocation of Resources Without Limits) both flag this as a critical resource management failure. The user loses their entire conversation state, and you have no log data to diagnose the spike.

Add a token-counting utility using tiktoken or js-tiktoken and measure the full payload before every API call. Store the count alongside the request in structured logs so you can see which sessions are approaching limits.

// src/lib/ai/token-counter.ts
import { encoding_for_model } from "tiktoken";

export function countMessageTokens(
  messages: Array<{ role: string; content: string }>,
  model: string = "gpt-4o"
): number {
  const encoder = encoding_for_model(model as Parameters<typeof encoding_for_model>[0]);
  let count = 4;
  for (const msg of messages) {
    count += 4;
    count += encoder.encode(msg.content).length;
  }
  encoder.free();
  return count;
}

Call this before streamText or generateText in src/app/api/chat/route.ts. Log the result and branch on whether you're within the model's limit minus a response reserve.