A sitemap tells search engines which pages you consider worth crawling. It helps discovery. It does not make pages rank, and it will not rescue a page Google has decided is not worth indexing.
An XML sitemap lists the URLs you want search engines to crawl, with the date each was last modified. Include only canonical, indexable pages that return a 200 status. Submit it in Search Console and reference it in robots.txt. It helps discovery, particularly on large or poorly linked sites, and it is not a ranking factor.
What belongs in it
- Canonical URLs only, never variants that canonicalise elsewhere.
- Pages returning a 200 status. No redirects, no 404s.
- Indexable pages only. Anything carrying noindex should not be listed.
- Accurate lastmod dates, which help crawlers prioritise recently changed pages.
- The https, correct-domain version of every URL.
A sitemap full of redirects and noindexed pages sends mixed signals: you are asking Google to crawl pages you have separately told it to ignore.
What it does and does not do
| Does | Does not |
|---|---|
| Help search engines discover pages | Guarantee indexing |
| Signal which pages you consider important | Improve rankings |
| Communicate when content changed | Fix thin or duplicate content |
| Help on large or poorly linked sites | Replace internal linking |
This is the most common misunderstanding. A page that is in the sitemap and still not indexed is being judged on its merits, and adding it again will not change that.
Setting it up
- Generate it automatically. Most CMS platforms do this natively or through a plugin.
- Keep it under 50,000 URLs and 50 MB per file, splitting into an index of multiple sitemaps if needed.
- Reference it in robots.txt with a Sitemap line.
- Submit it in Google Search Console and Bing Webmaster Tools.
- Check the coverage report afterwards for pages Google chose not to index and why.
Reading the coverage report
- Submitted and indexed: working as intended.
- Discovered, currently not indexed: Google knows about it and has not crawled it. Often a crawl budget or perceived quality signal.
- Crawled, currently not indexed: Google looked and decided against it. Usually thin, duplicate, or low value.
- Excluded by noindex: intentional if you meant it, a bug if you did not.
- Alternate page with proper canonical: normal, and should not be in your sitemap.
Crawled but not indexed is the one worth acting on. It means the page was assessed and rejected, which is a content problem rather than a technical one.
Frequently asked questions
Do I need an XML sitemap?
Most sites benefit, and large sites or sites with weak internal linking benefit most. A small, well-linked site will usually be crawled fully without one. It costs nothing to have and removes one possible reason for pages being missed.
Does a sitemap improve rankings?
No. It aids discovery, which is a prerequisite for ranking, but it is not a ranking factor. A page in your sitemap competes on the same terms as any other.
How often should a sitemap update?
Automatically, whenever content changes. Most CMS platforms handle this. Manually maintained sitemaps drift out of date quickly and end up listing pages that no longer exist.
Should I include every page?
Only canonical, indexable pages returning 200. Exclude noindexed pages, redirects, error pages and duplicates. Listing pages you have told Google to ignore sends contradictory signals.
My page is in the sitemap but not indexed. Why?
Being listed does not guarantee indexing. Google decides based on quality, uniqueness and perceived value. Check Search Console for the specific status. Crawled but not indexed means it was assessed and rejected, which is a content issue.