XML sitemap includes all indexable pages; noindex pages excluded

Pages missing from sitemap.xml rely entirely on internal link discovery — newly launched products, fresh blog posts, and deep category pages can sit unindexed for weeks. Conversely, including noindex pages in the sitemap sends contradictory signals that Google Search Console flags as Submitted URL marked noindex errors, which erode trust in the entire sitemap and slow crawl scheduling for legitimate URLs.

Generate the sitemap dynamically from the same data source that produces your routes so drift is impossible. Implement in app/sitemap.ts:

export default async function sitemap() {
  const products = await db.product.findMany({ where: { published: true } })
  return products.map(p => ({ url: `https://yoursite.com/products/${p.slug}`, lastModified: p.updatedAt }))
}

• ID: seo-advanced.crawlability.sitemap-completeness

• Severity: medium

• What to look for: Count all indexable public pages by scanning the route structure. Count all URLs in sitemap.xml (or sitemap index). Compare the two lists: enumerate pages missing from the sitemap and pages in the sitemap that have noindex meta tags. At least 95% coverage is required.

• Pass criteria: Sitemap includes at least 95% of indexable public pages. Zero noindex pages appear in the sitemap. Sitemap is valid XML with no parsing errors. Report: "Sitemap contains X of Y indexable pages (Z% coverage)."

• Fail criteria: Sitemap covers fewer than 95% of indexable pages, or at least 1 noindex page is included in the sitemap, or sitemap has XML parsing errors.

• Skip (N/A) when: No sitemap.xml or app/sitemap.ts exists (sitemap presence is covered by SEO Fundamentals).

• Detail on fail: "Sitemap contains 50 of 65 indexable pages (77% coverage) — missing /blog, /pricing, and 13 product pages" or "Sitemap includes 2 /admin pages which have noindex meta tags".

• Remediation: Regenerate sitemap to include all public routes. Use a dynamic sitemap in app/sitemap.ts:

  // app/sitemap.ts
  export default function sitemap() {
    return [
      { url: 'https://yoursite.com', lastModified: new Date() },
      { url: 'https://yoursite.com/about', lastModified: new Date() },
      // ... dynamically generated from your pages
    ]
  }