Sitemap URL is referenced in robots.txt
Why it matters
The Sitemap: directive in robots.txt is how crawlers you have never heard of — Bing, Yandex, DuckDuckGo, Perplexity, ChatGPT's crawler, and every GEO/AI indexer — discover your sitemap without you submitting it through a dashboard. Google Search Console covers Googlebot alone. If the directive is missing, those crawlers fall back to link discovery only, and any orphaned route never gets indexed outside Google. One line of config unlocks distribution across every search engine and AI system that obeys the standard.
Severity rationale
Medium because Google still finds the sitemap via Search Console, but non-Google crawlers and AI indexers miss it entirely.
Remediation
Append a Sitemap: line to your robots.txt with the absolute HTTPS URL of your sitemap. If you are on Next.js, create app/robots.ts so the directive stays in version control alongside the sitemap.
User-agent: *
Allow: /
Sitemap: https://yoursite.com/sitemap.xml
If you ship multiple sitemaps (for example, one per content type), include one Sitemap: directive per file — the standard allows multiple lines.
DetectionPro
The detection method for this pattern is part of AuditBuffet Pro.
The full method (what to look for, pass and fail conditions, and how results get reported) ships inside the SEO Fundamentals Audit. The pattern definition, severity rationale, and remediation remain openly licensed under CC-BY-4.0.
Taxons
History
- 2026-04-18·v1.0.0·Initial import from seo-fundamentals·automated