Variant assignment is deterministic per contact

Non-deterministic variant assignment means the same contact can land in variant A for the initial campaign send and variant B for a follow-up in the same experiment. This cross-contaminates both samples: contacts with mixed exposure invalidate the per-variant outcome metrics, making it impossible to declare a statistically valid winner. A/B test results built on drifting assignments generate false confidence — you ship the wrong variant because the data said it won, when in fact both groups were mixed. The data-integrity taxon is the direct failure mode: the experiment structure looks valid but the underlying sample is corrupt.

Hash contactId + experimentId to derive a stable bucket number — the same contact always maps to the same variant, across deploys and across send-time calls:

import { createHash } from 'crypto'

function assignVariant(
  contactId: string,
  experimentId: string,
  numVariants: number
): number {
  const hash = createHash('sha256')
    .update(`${experimentId}:${contactId}`)
    .digest('hex')
  return parseInt(hash.slice(0, 8), 16) % numVariants
}

// 0 = control, 1 = variant
const variant = assignVariant(contact.id, 'subject-line-q1-2026', 2)

// Optionally persist for auditability — but never change once written
await db.experimentAssignments.upsert({
  where: { contact_id_experiment_id: { contact_id: contact.id, experiment_id: 'subject-line-q1-2026' } },
  create: { contact_id: contact.id, experiment_id: 'subject-line-q1-2026', variant, assigned_at: new Date() },
  update: {}
})

Never use Math.random() at send time unless you persist the result immediately.

• ID: deterministic-variant-assignment

• Severity: critical

• What to look for: Examine how contacts are assigned to A/B test variants. Look for variant assignment logic — it must produce the same variant for the same contact ID every time (deterministic). Non-deterministic patterns include: assigning a random variant at send time without persisting the assignment, using Math.random() without seeding, or computing the variant at read time without a stable lookup. Deterministic patterns include: hashing the contact ID (or contact ID + experiment ID) to derive a stable bucket number, or storing the assignment in a database table on first exposure and looking it up thereafter.

• Pass criteria: Variant assignment uses a deterministic method: either (a) a hash of contact_id + experiment_id mapped to a bucket, or (b) a persisted assignment table that stores the variant for each contact+experiment pair on first assignment and never changes. Quote the actual assignment function or hash logic found in the codebase. Count all variant assignment call sites — at least 1 must use a deterministic method.

• Fail criteria: Math.random() or any non-seeded randomization is used at send time without persisting the result. A contact could be assigned to variant A in one send and variant B in a follow-up in the same experiment. Using a seeded random without persisting the assignment does not count as pass if the seed can change between deployments.

• Skip (N/A) when: The project does not run A/B tests.

• Detail on fail: Example: "Variant assignment uses Math.random() at send time with no persistence — contacts in follow-up sends within the same experiment may receive a different variant" or "No assignment storage found — variant computed fresh each time from non-deterministic source"

• Remediation: Use a hash-based deterministic assignment:

  import { createHash } from 'crypto'
  
  // Deterministic bucket assignment (0 to numVariants - 1)
  function assignVariant(contactId: string, experimentId: string, numVariants: number): number {
    const hash = createHash('sha256')
      .update(`${experimentId}:${contactId}`)
      .digest('hex')
    // Take first 8 hex chars as a 32-bit integer, mod by variant count
    const bucket = parseInt(hash.slice(0, 8), 16) % numVariants
    return bucket
  }
  
  // Usage: 0 = control, 1 = variant A
  const variant = assignVariant(contact.id, 'subject-line-test-jan-2026', 2)
  
  // Persist assignment for auditability (optional but recommended)
  await db.experimentAssignments.upsert({
    where: { contact_id_experiment_id: { contact_id: contact.id, experiment_id: 'subject-line-test-jan-2026' } },
    create: { contact_id: contact.id, experiment_id: 'subject-line-test-jan-2026', variant, assigned_at: new Date() },
    update: {} // Never change once assigned
  })