System prompt token count is reasonable

The system prompt is the fixed tax paid on every single API call. Embedding a full FAQ document, a multi-page policy, or a large JSON schema in the system prompt means paying for thousands of tokens on every request — including the trivial ones. At 10,000 requests per day on GPT-4o at $2.50 per million input tokens, a bloated 3,000-token system prompt costs roughly $75/day in fixed overhead before any user message token is counted. OWASP LLM10 and ISO 25010 performance-efficiency flag this as architectural waste that compounds with volume.

Move static reference material out of the system prompt into a retrieval layer. Inject only the content relevant to the current query, not the entire corpus.

// Before — entire FAQ hardcoded in system prompt (3000+ tokens per call)
const systemPrompt = `You are a helpful assistant. Here is our FAQ:
Q: How do I reset my password? A: ...
[50 more Q&A pairs]`;

// After — concise system prompt; relevant FAQ injected per-request via RAG
const systemPrompt = `You are a customer support assistant.
Answer based on the provided context only. Say so if the answer is not in the context.`;

const relevantContext = await retrieveRelevantFAQ(userMessage);

After restructuring, verify the system prompt is under 1,500 tokens by running it through countMessageTokens([{ role: "system", content: systemPrompt }], model) and logging the result.