A context compaction or summarization strategy is implemented for long sessions

Simple FIFO truncation discards messages with no regard for their semantic value. The user's name stated at message 1, a constraint given at message 5, or a decision made at message 10 vanishes permanently when it slides out of the window. In multi-step workflows — job application assistants, planning tools, tutoring systems — this causes visible regression: the AI contradicts itself or asks questions the user already answered. NIST AI RMF MANAGE 2.2 requires managing model limitations that affect outcomes; history amnesia is a direct consequence of ignoring this.

Implement a summarization step that fires when history exceeds a turn threshold. Use a cheap, fast model for the summarization call — not the frontier model you use for the primary task.

// src/lib/ai/compaction.ts
export async function compactHistory(
  messages: Array<{ role: string; content: string }>,
  threshold: number = 20
): Promise<Array<{ role: string; content: string }>> {
  if (messages.length <= threshold) return messages;

  const toSummarize = messages.slice(0, messages.length - 10);
  const recentMessages = messages.slice(messages.length - 10);

  const { text: summary } = await generateText({
    model: openai("gpt-4o-mini"),
    prompt: `Summarize this conversation, preserving key facts, names, goals, and decisions:\n\n${
      toSummarize.map(m => `${m.role}: ${m.content}`).join("\n")
    }`,
    maxTokens: 300,
  });

  return [
    { role: "system", content: `[Earlier conversation summary]: ${summary}` },
    ...recentMessages,
  ];
}

Verify by stating a specific fact at message 1, continuing for 25+ turns, then asking the AI to recall that fact — it should still know it.