Googlebot allocates a fixed crawl budget per site, and every request to a noindex admin or debug route is a request it did not spend on a page you actually want ranked. Noindex URLs in the sitemap send contradictory signals — submit this URL, but do not index it — which Google's Search Console flags as a coverage error and which slows indexation of legitimate new content.
Info because crawl-budget inefficiency mostly affects large sites and rarely suppresses individual page rankings directly.
Filter noindex pages out of the sitemap generator and remove links to consistently-noindex routes from the public navigation and footer. Scope admin/dashboard links to authenticated sessions so they never appear in public HTML. Verify the result by requesting the sitemap and confirming zero noindex URLs are listed.
// app/sitemap.ts
const pages = allPages.filter(p => !p.noindex)
return pages.map(p => ({ url: p.url, lastModified: p.updatedAt }))
ID: marketing-advanced-seo.technical-seo.crawl-budget-efficiency
Severity: info
What to look for: Count all noindex pages that appear in the sitemap or are linked from navigation. Enumerate wasted crawl paths. Check whether pages or sections marked as noindex are still reachable via internal links from indexable pages. Look for: (1) authenticated/admin route sections (e.g., /admin/*, /dashboard/*) that have noindex but are linked from public pages, (2) utility pages (/api-docs, /debug) that are noindex but appear in the site navigation or footer, (3) pages with noindex directives that appear as URL entries in the XML sitemap (contradictory signals).
Pass criteria: No-index pages are not linked from indexed pages (or are linked only from pages that are themselves noindex), and no more than 0 noindex pages appear in the XML sitemap. Zero noindex pages should be included in the sitemap or linked from primary navigation.
Fail criteria: Public indexed pages contain links to consistently-noindex pages. OR noindex pages appear as URL entries in the XML sitemap.
Skip (N/A) when: No pages with noindex directives detected in the project.
Cross-reference: For sitemap completeness, see the seo-advanced audit sitemap-completeness check.
Detail on fail: "Crawl budget inefficiency: /admin and /api-docs are set noindex but are linked from the public footer. These pages consume crawl budget without contributing to indexation. Also: /api-docs appears in sitemap.xml despite being noindex."
Remediation: For large sites, crawl budget matters. For smaller sites, this is lower priority but still worth cleaning up. Remove links to consistently-noindex pages from your public navigation. Remove noindex URLs from your sitemap. If admin or tool pages must appear in the nav, consider scoping those links to authenticated sessions only.
// app/sitemap.ts — exclude noindex pages
const pages = allPages.filter(p => !p.noindex)
return pages.map(p => ({ url: p.url, lastModified: p.updatedAt }))