Circuit breaker on repeated ESP failures
Why it matters
Exponential backoff slows down retry frequency, but it does not stop new jobs from being accepted and attempted against a broken ESP. A circuit breaker adds the missing control: after N consecutive failures, it stops attempting sends entirely and waits for a cooldown before probing again. Without it, a prolonged ESP outage fills the queue with failed jobs that burn retry budget. CWE-400 applies because new jobs continue to consume processing resources against an endpoint that is confirmed unavailable. The circuit breaker is the difference between "we paused sends during the outage" and "we burned through all retry attempts before the outage resolved."
Severity rationale
Low because exponential backoff provides partial protection, but without a circuit breaker, indefinite new-job ingestion during an outage still exhausts retry budget and worker capacity.
Remediation
Wrap the ESP send call with a circuit breaker using a library like opossum in your send adapter (e.g., src/lib/email/providers/sendgrid.ts):
import CircuitBreaker from 'opossum'
const breaker = new CircuitBreaker(esp.send.bind(esp), {
timeout: 5000,
errorThresholdPercentage: 50,
resetTimeout: 60_000 // 1 minute cooldown
})
breaker.on('open', () => {
alertOps('ESP circuit breaker opened — sends paused for 60s', 'critical')
})
export const sendEmail = (msg: EmailMessage) => breaker.fire(msg)
This check may be skipped when ab-001981 (ESP failover) is already passing, as immediate failover to a secondary fills the circuit breaker role.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the Operational Resilience (Email) Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- cwe · CWE-400 — Uncontrolled Resource Consumption — indefinite retries against broken ESP without circuit breaker
- iso-25010:2011 · reliability.fault-tolerance — Reliability / Fault Tolerance — circuit breaker prevents cascading failures during prolonged ESP outage
Taxons
History
- 2026-04-18·v1.0.0·Initial import from operational-resilience-email·automated