Repeated or identical AI queries are cached

Many AI applications have query patterns where the same prompt runs repeatedly across different users: SEO description generation for a product page, FAQ answer retrieval for a support bot, or summarization of a static document. Without a cache layer, each of these requests hits the API and bills you for the same tokens repeatedly. On a product listing page visited by 10,000 users per day with identical AI-generated descriptions, you pay for 10,000 API calls when 1 would suffice. ISO 25010 performance-efficiency requires eliminating redundant computation at scale.

Wrap AI calls in a cache layer keyed by a SHA-256 hash of the messages and model. Include the model name in the cache key — model upgrades must invalidate stale responses.

// src/lib/ai/cached-generate.ts
import { createHash } from "crypto";
import { redis } from "@/lib/redis";

export async function cachedGenerate(
  messages: Array<{ role: string; content: string }>,
  options: { model: string; maxTokens: number; ttlSeconds?: number }
): Promise<string> {
  const cacheKey = `ai:${createHash("sha256")
    .update(JSON.stringify({ messages, model: options.model }))
    .digest("hex")
    .slice(0, 16)}`;

  const cached = await redis.get<string>(cacheKey);
  if (cached) return cached;

  const { text } = await generateText({
    model: openai(options.model),
    messages,
    maxTokens: options.maxTokens,
  });

  await redis.set(cacheKey, text, { ex: options.ttlSeconds ?? 3600 });
  return text;
}

Verify by making the same request twice and confirming the second response is near-instant with a cache hit visible in logs.