Expensive operations are cached or marked as fresh-required

LLM API calls are the most expensive per-request operations in a modern web stack — GPT-4o costs $2.50/M input tokens and $10/M output tokens, and a single summarization endpoint with no caching pays that cost on every request, even for identical inputs. OWASP LLM10 (Unbounded Consumption) specifically identifies missing caching as a cost-amplification risk for LLM-backed applications. At modest scale, 1,000 daily requests to an uncached summarization endpoint can cost $50–$200/day in token charges. The fix is additive — unstable_cache wraps any async function — and reduces cost to near-zero for duplicate inputs without changing the API contract.

Wrap the LLM call in unstable_cache (Next.js) or a Redis-backed cache keyed to a hash of the input. For deterministic inputs, a 24-hour TTL is usually appropriate.

import { unstable_cache } from 'next/cache'
import { createHash } from 'crypto'
import OpenAI from 'openai'

const openai = new OpenAI()

const cachedSummarize = unstable_cache(
  async (text: string) => {
    const res = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: `Summarize: ${text}` }],
    })
    return res.choices[0]?.message.content ?? ''
  },
  ['summarize'],
  { revalidate: 86400 },
)

export async function POST(req: Request) {
  const { text } = await req.json()
  const summary = await cachedSummarize(text)
  return Response.json({ summary })
}

For the AI Token Optimization audit (ai-token-optimization), which covers token usage patterns in depth, see the dedicated audit.