# XML sitemap includes all indexable pages; noindex pages excluded

- **Pattern:** `ab-002493` (`seo-advanced.crawlability.sitemap-completeness`)
- **Severity:** medium
- **Lifecycle:** active
- **Last modified:** 2026-04-18
- **Canonical URL:** https://auditbuffet.com/patterns/ab-002493
- **License:** CC-BY-4.0 — attribute to AuditBuffet Pattern Catalog (https://auditbuffet.com/patterns/ab-002493)

## Why it matters

Pages missing from `sitemap.xml` rely entirely on internal link discovery — newly launched products, fresh blog posts, and deep category pages can sit unindexed for weeks. Conversely, including `noindex` pages in the sitemap sends contradictory signals that Google Search Console flags as `Submitted URL marked noindex` errors, which erode trust in the entire sitemap and slow crawl scheduling for legitimate URLs.

## Severity rationale

Medium because sitemap drift delays indexation but internal linking partially compensates for missing entries.

## Remediation

Generate the sitemap dynamically from the same data source that produces your routes so drift is impossible. Implement in `app/sitemap.ts`:

```ts
export default async function sitemap() {
  const products = await db.product.findMany({ where: { published: true } })
  return products.map(p => ({ url: `https://yoursite.com/products/${p.slug}`, lastModified: p.updatedAt }))
}
```

## Detection

- **ID:** `sitemap-completeness`
- **Severity:** `medium`
- **What to look for:** Count all indexable public pages by scanning the route structure. Count all URLs in `sitemap.xml` (or sitemap index). Compare the two lists: enumerate pages missing from the sitemap and pages in the sitemap that have `noindex` meta tags. At least 95% coverage is required.
- **Pass criteria:** Sitemap includes at least 95% of indexable public pages. Zero noindex pages appear in the sitemap. Sitemap is valid XML with no parsing errors. Report: "Sitemap contains X of Y indexable pages (Z% coverage)."
- **Fail criteria:** Sitemap covers fewer than 95% of indexable pages, or at least 1 noindex page is included in the sitemap, or sitemap has XML parsing errors.
- **Skip (N/A) when:** No `sitemap.xml` or `app/sitemap.ts` exists (sitemap presence is covered by SEO Fundamentals).
- **Detail on fail:** `"Sitemap contains 50 of 65 indexable pages (77% coverage) — missing /blog, /pricing, and 13 product pages"` or `"Sitemap includes 2 /admin pages which have noindex meta tags"`.
- **Remediation:** Regenerate sitemap to include all public routes. Use a dynamic sitemap in `app/sitemap.ts`:

  ```ts
  // app/sitemap.ts
  export default function sitemap() {
    return [
      { url: 'https://yoursite.com', lastModified: new Date() },
      { url: 'https://yoursite.com/about', lastModified: new Date() },
      // ... dynamically generated from your pages
    ]
  }
  ```

Taxons: findability

HTML version: https://auditbuffet.com/patterns/ab-002493