Batch processing is used for non-interactive AI tasks

The OpenAI Batch API offers a 50% cost discount and higher throughput limits for requests that can tolerate a 24-hour completion window. Non-interactive tasks — nightly content tagging, bulk summarization, product description generation, weekly report creation — have no reason to use real-time standard API calls and pay the full rate. Sending these jobs one-by-one through the synchronous endpoint forgoes half the available cost savings and burns synchronous rate limit capacity that user-facing features need. ISO 25010 performance-efficiency requires that resource consumption be calibrated to actual operational constraints.

Migrate background AI jobs to the OpenAI Batch API using a JSONL request file. The 50% discount applies automatically; standard rate limits do not apply to batch jobs.

// src/jobs/bulk-summarize.ts
const batchRequests = articlesToSummarize.map(article => ({
  custom_id: `article-${article.id}`,
  method: "POST",
  url: "/v1/chat/completions",
  body: {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `Summarize: ${article.content}` }],
    max_tokens: 200,
  },
}));

const filePath = "/tmp/batch-requests.jsonl";
fs.writeFileSync(filePath, batchRequests.map(r => JSON.stringify(r)).join("\n"));

const file = await openai.files.create({ file: fs.createReadStream(filePath), purpose: "batch" });
await openai.batches.create({ input_file_id: file.id, endpoint: "/v1/chat/completions", completion_window: "24h" });

Verify by confirming that cron jobs and queue workers route through openai.batches.create rather than chat.completions.create.