- The Adventures of SEO Boy® - http://www.seoboy.com -

Need Another Reason to Use XML Sitemaps? Compare Index Stats in Google Webmaster Tools

Posted By John On January 29, 2009 @ 4:59 pm In Crawlability | 3 Comments

As we’ve discussed before, submitting an XML Sitemap [1] to Google (and the other search engines) is an openly debated practice. Proponents of sitemaps tout benefits to indexation and visibility with the search engines. Those against hold a more principled stance – that your site should be optimized in such a way that submitting an XML Sitemap isn’t needed. I for one fall under the proponent category, and I’ve found another reason to back up my case: Google Webmaster Tools’ Sitemap Details page. In addition to giving you details on when you first submitted your sitemap, when it was last downloaded and its current status (“OK”), this details page offers an interesting look at how well your content is being indexed [2] compared to the data submitted via sitemap via the “Indexed URLs in Sitemap” statistic:

Everyone knows how (or should know how) to use the site: command to review pages indexed in Google. This is the primary method for recognizing how well, or how deeply your site is being crawled and indexed. But this Sitemap Details information adds a new stat into the mix, one that I’m discovering with a mix of satisfaction and curiosity. What exactly does this stat mean? I’m not 100% certain, but I’ve got a few ideas of what you can do with the information.

  1. This statistic can give you a quick appraisal of your overall performance in terms of indexation. Check the Sitemap Details page regularly to keep tabs of how well you’re doing.
  2. Use this as a diagnostic tool. If you find that your website traffic has dropped, and you’re unsure what’s to blame – check to see if your sitemap-to-index ratio has dropped. This could mean that despite having your URLs in the sitemap, there’s a crawlability issue on your site that needs fixed.
  3. Regardless of performance, this statistic could be the impetus to sniff out crawlability issues on your website. If you find that your sitemap-to-index ratio is always down, you’ve got work to do.

I should point out that Google specifically states [3] that this data is “a close approximation of the status of your URLs” and that “this figure might not be 100%.” However, when I checked the stats for SEO Boy today, the number of URLs listed in GWT for my XML Sitemap was 120 after having been downloaded an hour prior – and 120 was right on. My “Indexed URLs in Sitemap” was 115 – which means I’ve got some work to do!


Article printed from The Adventures of SEO Boy®: http://www.seoboy.com

URL to article: http://www.seoboy.com/need-another-reason-to-use-xml-sitemaps-compare-index-stats-in-google-webmaster-tools/

URLs in this post:

[1] submitting an XML Sitemap: http://www.seoboy.com/my-favorite-seo-argument-sitemap-vs-no-sitemap/

[2] content is being indexed: http://www.seoboy.com/get-hip-to-seo-lingo-by-distinguishing-between-crawl-index-and-cache/

[3] Google specifically states: https://www.google.com/support/webmasters/bin/answer.py?answer=80488&ctx=tltp&hl=en

[4] How and Why You Should be Using Google Webmaster Tools – Part 2: http://www.seoboy.com/how-and-why-you-should-be-using-google-webmaster-tools-part-2/

[5] How and Why You Should be Using Google Webmaster Tools – Part 1: http://www.seoboy.com/how-and-why-you-should-be-using-google-webmaster-tools-part-1/

[6] How to Set Up and Use MSN’s Live Search: Webmaster Central: http://www.seoboy.com/how-to-set-up-and-use-msns-live-search-webmaster-central/

Copyright © 2008 The Adventures of SEO Boy. All rights reserved.