Malformed records quarantined, not silently discarded
Why it matters
Silently discarding records that fail validation destroys forensic visibility into data quality problems. A catch block that logs an error and continues leaves no persistent artifact of what was rejected, why, or how many records were affected. CWE-390 (Detection of Error Condition Without Action) applies directly — the error is detected but no remediation path exists. ISO 25010:2011 §4.2.5 (reliability) requires that fault conditions produce recoverable states. Without a quarantine store, a systematic problem in a purchased list (wrong column mapping, encoding issue) will produce thousands of silent discards that are invisible until someone notices the list's expected contact count does not match what was inserted.
Severity rationale
Medium because silent discards make systematic ingestion failures invisible, prevent reprocessing when the root cause is fixed, and violate the principle that all received data should have an auditable disposition.
Remediation
Create a contact_quarantine table and write every rejected record to it with the raw payload and rejection reason preserved. Never discard a record that has been received by your ingestion pipeline.
CREATE TABLE contact_quarantine (
id TEXT PRIMARY KEY DEFAULT gen_random_uuid(),
raw_payload JSONB NOT NULL,
rejection_reason TEXT NOT NULL,
quarantined_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
source_id TEXT,
reviewed BOOLEAN NOT NULL DEFAULT FALSE
);
// src/lib/ingestion/quarantine.ts
export async function quarantine(
raw: unknown,
reason: string,
sourceId?: string
) {
await db.contactQuarantine.create({
data: {
raw_payload: raw as Prisma.InputJsonValue,
rejection_reason: reason,
source_id: sourceId ?? null,
},
})
}
Add a reviewed flag and a simple admin interface so operations teams can inspect, correct, and reprocess quarantined records.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the Data Sourcing & Provenance Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
External references
- cwe · CWE-390 — Detection of Error Condition Without Action
- iso-25010:2011 · reliability.availability — Reliability — malformed records should not be silently lost
Taxons
History
- 2026-04-18·v1.0.0·Initial import from data-sourcing-provenance·automated