Rollback procedure executed successfully in staging or production within last 3 months

A rollback procedure that exists on paper but has never been executed is not a rollback procedure — it is a hypothesis. SOC 2 A1.3 and NIST CP-4 require that recovery procedures be tested, not just documented. Teams that have never exercised rollback discover gaps (missing permissions, undocumented dependencies, wrong migration commands) during an active production incident, extending recovery time from minutes to hours. ISO 25010 reliability.recoverability requires verified recovery capability, not theoretical capability.

Schedule a rollback drill and document it in your DEPLOYMENT.md or an incident log file.

# Rollback Drills

## 2026-02-15 — Quarterly Drill
- Deployed v2.1.0 to production
- Triggered rollback to v2.0.9
- Time to rollback: 8 minutes
- Migration rollback succeeded: yes
- Health checks passed: yes
- Gaps found: SSH key for prod DB was missing from on-call docs (fixed)
- Status: PASSED

Run drills monthly or quarterly depending on deployment frequency. Treat gaps found during the drill as P2 action items — document and fix them before the next drill. The goal is a rollback that takes under 15 minutes with any engineer on-call, not just the one who wrote the procedure.

• ID: deployment-readiness.monitoring-alerting.rollback-tested

• Severity: medium

• What to look for: Enumerate every relevant item. Look for documentation of a recent rollback test or incident. Check DEPLOYMENT.md, incident logs, or pull requests with evidence of a rollback execution. The test should have been within the last 3 months.

• Pass criteria: Evidence exists of a successful rollback test or production incident rollback within the last 3 months. Rollback time was measured and documented.

• Fail criteria: No evidence of a rollback test or execution, or last rollback was more than 3 months ago.

• Skip (N/A) when: The project is not planned for production, or has been in production for less than 3 months.

• Detail on fail: "No evidence of a rollback test or execution. Last rollback (if any) was over 3 months ago." or "Rollback procedure documented but never executed or tested."

• Remediation: Schedule a rollback drill. Document it in your incident log or in a comment in DEPLOYMENT.md:

  # Incident Log / Rollback Drills
  
  ## 2026-02-15: Rollback Drill
  - Deployed version X to production
  - Triggered rollback to version X-1
  - Time to rollback: 8 minutes
  - All health checks passed
  - Status: ✓ Successful
  

  Then test your rollback procedure regularly (monthly or quarterly).