Production pages are not accidentally set to noindex

An unconditional noindex tag is the single most destructive SEO defect possible: it removes the page from every search engine, permanently, until the directive is lifted and the page is recrawled. Common failure modes include a staging-site robots: { index: false } copied to production, a tutorial snippet left in app/layout.tsx that poisons every child route, and CMS preview flags that leak into published content. Recovery is not instant — even after removing the tag, pages often take weeks to return to prior rankings.

Audit every metadata.robots export and every <meta name="robots"> tag for unconditional noindex or { index: false } values. Gate any legitimate non-production noindex on process.env.NODE_ENV or a deployment-environment check — never a static false.

// Correct: only noindex when NOT in production
export const metadata = {
  robots: {
    index: process.env.NODE_ENV === 'production',
  },
}

Also inspect next.config.js headers and any middleware setting X-Robots-Tag: noindex — those override metadata and are easy to miss.

• ID: seo-fundamentals.discoverability.no-noindex-production

• Severity: critical

• What to look for: Search for noindex in meta tags across all page routes. Check for <meta name="robots" content="noindex">, metadata.robots.index: false, or X-Robots-Tag headers in config. Be aware of environment-conditional noindex (acceptable — only development/staging should be noindex, not production).

• Pass criteria: List all occurrences of noindex across all page routes, metadata exports, and robot directives. No more than 0 production page routes may have unconditional noindex directives — any unconditional noindex is a fail. Environment-conditional noindex (that only applies in development/staging) is acceptable. When a noindex directive is gated on an environment variable, pass if the condition resolves to "do not noindex" when NODE_ENV === 'production'. For example: robots: { index: process.env.NODE_ENV === 'production' } passes because index is true in production. If the conditional logic is complex or unclear (e.g., a custom env var with unknown deployment values), still pass but include a note in the detail field: "Conditional noindex detected — verify it resolves correctly in production."

• Fail criteria: Any page route has an unconditional noindex meta tag or robots directive that would apply in production. Report: "Found X unconditional noindex directives across Y page routes."

• Skip (N/A) when: Never — accidental noindex is a critical SEO issue for any public site.

• Detail on fail: Name the pages with noindex. Example: "Unconditional noindex found on /blog layout — all blog pages would be hidden from search engines" or "Root layout sets robots: { index: false } with no environment condition"

• Remediation: A noindex directive tells search engines not to show your page in search results. This is appropriate during development but must not be present in production. Check for and remove any unconditional noindex:

  // Remove this from production pages:
  export const metadata = {
    robots: { index: false }  // This hides the page from search engines
  }
  

  If you need noindex in development only, use an environment check:

  export const metadata = {
    robots: {
      index: process.env.NODE_ENV === 'production',
    },
  }