Every Campaign Analytics & Attribution Audit check

Apple Mail Privacy Protection (MPP), rolled out in iOS 15, proxy-loads tracking pixels through Apple CDN servers the instant an email arrives — before the recipient ever opens it. This inflates measured open rates by 30–60% for audiences with significant iOS or Apple Mail usage, silently corrupting the engagement signals your send-time optimization, list segmentation, and campaign ROI calculations depend on. Campaign decisions made on these numbers — suppressing disengaged contacts, picking a winning A/B variant by open rate, or attributing revenue to high-open-rate sequences — all degrade in quality. The data-integrity taxon is directly implicated: the metric exists but measures something fundamentally different from human engagement.

Why this severity: Critical because MPP distortion silently corrupts every downstream metric that uses open rate as an input — A/B test results, segmentation criteria, engagement scoring — producing confident but wrong decisions at scale.

Shared ESP tracking domains like click.sendgrid.net and email.mailgun.net pool their reputation across every tenant on the platform, so one spammer's blasts drag your click-throughs into spam folders and Safe Browsing warnings. Owned redirect subdomains tie click reputation to your sending domain, keep DKIM/SPF alignment intact, survive ESP migrations without breaking historical links, and stop Gmail and Outlook from flagging third-party hostnames as phishing redirects in the user's browser.

Why this severity: High because a poisoned shared tracking domain suppresses deliverability and click-through across every campaign until migration completes.

Open-tracking pixels only fire when the receiving client treats the endpoint as a loadable image — a 204 response, a redirect, or an `image/gif` Content-Type served as `text/html` causes Gmail, Outlook, and Apple Mail to silently drop the request, so your open-rate metric reports zero even when the email was read. Cache-allowed pixels compound the damage by suppressing re-open events, which breaks engagement scoring, re-engagement workflows, and the funnel attribution data that downstream CAC calculations rely on.

Why this severity: Medium because broken pixels corrupt engagement metrics and re-engagement targeting without directly affecting deliverability or revenue.

Click redirect handlers sit in the path between every tracked link in your emails and the page the recipient is trying to reach. When tracking logic throws an unhandled exception — a database timeout, a network blip, a missing link ID — and the redirect doesn't fire, the user hits a 500 error instead of your landing page. Every tracked link in every sent email becomes a broken link for the duration of the outage. This is an iso-25010:2011 reliability failure: the observability instrumentation should never degrade the user's ability to navigate to the destination URL.

Why this severity: High because a tracking write failure converts every email link into a broken link for the user, destroying the campaign's conversion path and damaging sender reputation in a way that affects every recipient simultaneously.

Storing email events as daily aggregate counts instead of timestamped rows destroys information that cannot be reconstructed later. Time-of-day open patterns, send-time optimization, session attribution, and sequence latency analysis all require sub-day precision. A schema that records `open_count = 47` for a given date tells you nothing about whether those opens happened at 7am or 11pm, making it impossible to determine optimal send windows for your audience or to join open events to downstream conversion events by session. The data-integrity taxon names the exact failure: you have a metric, but you've discarded the resolution needed to act on it.

Why this severity: Low because daily aggregates still support basic campaign comparison reporting, but they permanently foreclose time-of-day analysis and session-level attribution that become meaningful at scale.

Non-deterministic variant assignment means the same contact can land in variant A for the initial campaign send and variant B for a follow-up in the same experiment. This cross-contaminates both samples: contacts with mixed exposure invalidate the per-variant outcome metrics, making it impossible to declare a statistically valid winner. A/B test results built on drifting assignments generate false confidence — you ship the wrong variant because the data said it won, when in fact both groups were mixed. The data-integrity taxon is the direct failure mode: the experiment structure looks valid but the underlying sample is corrupt.

Why this severity: Critical because non-deterministic assignment silently corrupts experiment samples, producing statistically invalid results that lead to deploying the wrong variant with false confidence.

Launching an A/B test without calculating required sample size means you have no principled stopping rule. Teams peeking at early results and declaring a winner at 200 contacts when 800 were needed inflate false positive rates from 5% to 25% or higher — a phenomenon called peeking bias. You ship subject lines, CTAs, or send times based on noise rather than signal. In iso-25010:2011 terms this is a functional-suitability failure: the experiment tooling exists but cannot reliably distinguish a real effect from random variance. The Campaign Orchestration & Sequencing Audit covers how these flawed signals propagate into branching decisions.

Why this severity: High because experiments evaluated without a pre-calculated sample size routinely produce false positives, causing teams to deploy losing variants with statistical confidence that the system never actually earned.

Declaring a winner by comparing raw open or click rates without computing a p-value is the statistical equivalent of calling a coin biased after three flips. Even with reasonable sample sizes, peeking at running totals and picking the leader inflates your false positive rate dramatically — a standard 95% confidence threshold applied to a peeked result may provide only 70% actual confidence. You ship the wrong variant and attribute revenue lift to a change that was noise. In iso-25010:2011 terms this is a functional-suitability gap: the experiment infrastructure appears to work but cannot fulfill its core promise of reliable winner selection.

Why this severity: High because selecting winners without significance testing produces false positives at rates far above the stated confidence level, systematically directing future campaigns toward variants that never actually outperformed.

When a contact is simultaneously enrolled in a subject-line test and a send-time test that both use open rate as the outcome metric, the results of each experiment are contaminated by the other. You cannot isolate the effect of the subject line from the effect of the send time. Reported lifts from both experiments are artifacts of the interaction, not the individual treatments. In practice this leads to shipping changes that performed well in a contaminated test but have no effect or a negative effect in production. The data-integrity failure is structural: the experiment framework appears to produce results, but those results are not attributable to the variable being tested.

Why this severity: Medium because interaction contamination degrades experiment validity incrementally — results are not entirely wrong but cannot be trusted for the shared metric, leading to compounding misattribution across multiple experiment cycles.

An auto-winner mechanism that re-assigns control contacts to the leading variant partway through an experiment corrupts the experiment's results in two ways: it reduces the control group size mid-run (inflating variance) and introduces a survivor bias into the final comparison. Contacts who remained in the control group through the auto-switch may be systematically different from those who were switched. If the experiment hasn't reached required sample size when the auto-switch fires, you've now lost the ability to recover a clean result. You've shipped a variant based on preliminary data that may reverse at full sample — the classic peeking outcome — with no way to rerun the experiment cleanly.

Why this severity: High because premature control-group dissolution forfeits the experiment's ability to produce a valid result, and the corrupted data still looks like a conclusive win, making the failure invisible without a methodological audit.

Ad-hoc UTM concatenation produces `utm_source=Email`, `utm_source=email`, and `utm_source=newsletter` as three separate traffic sources inside GA4, Mixpanel, or Amplitude, which fragments every campaign comparison, corrupts channel-grouping rules, and prevents the conversion-linked-to-send join from matching rows. Inconsistent `utm_campaign` slugs also break the attribution model's joins against the sends table, so CAC and ROAS calculations understate email's contribution and misdirect budget toward channels with cleaner naming discipline.

Why this severity: High because inconsistent UTMs silently fragment attribution data and misroute channel spend decisions.

Pure last-click attribution credits 100% of every conversion to whichever email the contact happened to click immediately before purchasing, ignoring the nurture sequence, re-engagement email, and product-update announcement that actually moved them down the funnel. Finance then starves the campaigns doing the real work, marketing kills the sequences that compound over weeks, and the attribution table cannot answer basic retrospective questions like "which first-touch emails drive the highest LTV cohorts." Full touchpoint storage is the prerequisite for every other model.

Why this severity: High because last-click-only attribution systematically misdirects marketing budget away from the sequences that actually drive conversions.

Without a foreign key or UTM-derived reference linking conversion records to the originating email send, you cannot compute campaign ROI, per-campaign conversion rates, or the revenue contribution of any individual sequence or A/B test. You can observe that a purchase happened, but you cannot answer which campaign drove it. GDPR Art. 5(1)(a) (lawfulness, purpose limitation) and CCPA §1798.120 (right to opt out of sale tied to identifiable data processing) both implicitly require that you understand why you hold each piece of customer data — including behavioral data collected via ePD Art. 5(3) tracking pixels. Blind conversion logs satisfy none of these requirements and make all email program measurement impossible.

Why this severity: Critical because unlinked conversion data makes it impossible to compute campaign ROI, evaluate A/B test outcomes on revenue metrics, or attribute any business result to an email program — the entire analytics layer loses its core function.

Attribution windows determine how far back from a conversion you look for a causal email touchpoint. A 7-day window and a 30-day window produce materially different revenue attribution numbers for the same campaign — especially for products with long consideration cycles. If the window is a magic number hard-coded in four different query files, changing your attribution model to match your actual sales cycle requires a multi-file search-and-replace with no guarantee you caught every instance. This is an iso-25010:2011 maintainability failure: the system cannot adapt to a known parameter without engineering effort and regression risk.

Why this severity: Low because incorrect window values degrade attribution accuracy but don't cause data loss or security exposure — the risk is misleading reporting rather than a broken system.

Without a join that connects sends, opens, clicks, and conversions in a single query, every funnel question becomes a manual spreadsheet exercise — analysts export CSVs from three tables, paste them together, and produce metrics that drift between reports. Revenue-per-email-sent, send-to-conversion rate, and cost-per-acquisition cannot be computed reliably, which blocks executive reporting, A/B test readout, and the send-cadence decisions that depend on knowing which campaigns actually generate incremental revenue per delivered message.

Why this severity: Low because the underlying events are captured; only the reporting join layer is missing and can be added without data backfill.

Pre-aggregated campaign reports without a defined refresh schedule show users numbers that may be hours or days out of date — but without an 'as of' timestamp, users have no way to know. A campaign manager who reads an open rate of 18% at 3pm and acts on it may be looking at data last refreshed at 6am. Decisions about pausing underperforming campaigns, triggering follow-up sequences, or adjusting send frequency are all time-sensitive. The iso-25010:2011 reliability taxon is the exact failure: the system appears operational but does not guarantee freshness, making it unreliable for time-sensitive decisions.

Why this severity: Medium because stale reporting data delays actionable decisions by hours or days, but does not cause data loss or security exposure — the harm accumulates as missed optimization windows rather than immediate failures.

Incrementally-updated aggregate counters — open_count bumped by a database trigger on each incoming webhook — diverge from the raw event log whenever a webhook is delivered twice, a contact is suppressed after the fact, or a timezone offset causes an event to land in the wrong reporting bucket. Because the aggregate and raw events are now out of sync, every metric derived from the aggregate (open rate, CTR, RPE) is wrong by an unknown amount. The iso-25010:2011 functional-suitability taxon names this precisely: the reporting function exists but produces systematically incorrect values with no mechanism to detect or correct the drift.

Why this severity: High because incremental aggregate drift is silent — reports display confidently wrong numbers indefinitely, and the discrepancy compounds across campaign cohorts until a manual audit surfaces it.

SQL date bucketing with `DATE(occurred_at)` uses PostgreSQL's session timezone — typically UTC — without advertising that fact to users. A campaign sent at 11pm EST fires tracking events that land on the following UTC day, making the performance report look like the campaign ran a day later than it did. For teams optimizing by day-of-week or comparing week-over-week performance, these bucketing artifacts corrupt trend analysis. The data-integrity taxon names the failure: the data is accurate at the row level but the aggregation step introduces systematic misclassification that is invisible without timezone awareness.

Why this severity: Low because UTC-implicit bucketing distorts day-level aggregates rather than corrupting individual event records, limiting the failure to trend and time-of-day analysis rather than fundamental data integrity.

Raw email event data — individual timestamped rows for each open, click, bounce, and unsubscribe — is the foundation for attribution debugging, engagement scoring, send-time optimization, and retrospective analysis when you change attribution models. Cron jobs that prune events older than 30 days in favor of aggregate counts permanently destroy that foundation. You cannot reconstruct which contacts engaged during a specific window, rerun historical attribution with a different model, or answer 'why did this campaign's conversion rate differ from last quarter?' GDPR Art. 5(1)(e) (storage limitation) requires a documented retention policy — the absence of one is itself a compliance gap, not just an analytics deficiency.

Why this severity: Low because short retention degrades analytical capability incrementally rather than breaking current reporting, but the data loss is irreversible — you cannot recover deleted events when you later need them for retrospective analysis.