Validation results cached with TTL

DNS MX lookups and external validation API calls have measurable latency — typically 50–500ms per call — and most providers impose rate limits (e.g., 100 calls/minute on free tiers). Without caching, every signup and every re-validation check triggers a live network round-trip for the same domain. At 1,000 signups/hour from domains like gmail.com or company.com, redundant lookups waste budget, add latency to the user-facing signup flow, and risk triggering rate-limit errors that silently degrade validation coverage when the lookup fails open.

Cache MX and disposable-check results at the domain level using Redis with a TTL of at least 1 hour:

import { redis } from '@/lib/redis'

async function getCachedMxResult(domain: string): Promise<boolean | null> {
  const cached = await redis.get(`mx:${domain}`)
  return cached === null ? null : cached === '1'
}

async function setCachedMxResult(domain: string, valid: boolean): Promise<void> {
  await redis.setex(`mx:${domain}`, 6 * 60 * 60, valid ? '1' : '0')
}

// In the ingest handler:
const domain = email.split('@')[1]
let valid = await getCachedMxResult(domain)
if (valid === null) {
  valid = await hasValidMx(email)
  await setCachedMxResult(domain, valid)
}

If Redis is not available, a DB table with an expires_at column works as a fallback. A TTL of 1–24 hours is standard — MX records change infrequently.

• ID: data-quality-list-hygiene.email-validation.validation-cache

• Severity: low

• What to look for: Count all validation-related external calls (DNS MX lookup, API validation, disposable check) and for each, check whether results are cached. Performing a live DNS lookup or an external API call for every email on every operation is slow and unnecessary. Look for a cache layer (Redis, in-memory Map, or a domain_validations table with an expires_at column).

• Pass criteria: Domain-level validation results (MX, catch-all status, disposable status) are cached with a TTL of at least 1 hour. Repeated checks for the same domain within the TTL do not trigger a new lookup.

• Fail criteria: Every email validation triggers a live DNS lookup or API call with no caching, or cache TTL is less than 5 minutes.

• Skip (N/A) when: Validation is performed exclusively via a third-party bulk validation service during import (not in the real-time ingest path).

• Detail on fail: Example: "Every signup triggers a live DNS lookup with no result caching — high latency and rate limit risk"

• Remediation: Cache domain validation at a separate domain level:

  import { redis } from '@/lib/redis'
  
  async function getCachedMxResult(domain: string): Promise<boolean | null> {
    const cached = await redis.get(`mx:${domain}`)
    return cached === null ? null : cached === '1'
  }
  
  async function setCachedMxResult(domain: string, valid: boolean): Promise<void> {
    // Cache for 6 hours
    await redis.setex(`mx:${domain}`, 6 * 60 * 60, valid ? '1' : '0')
  }