Welcome to SEO Boy, the authority on search engine optimization -- how to articles, industry news, insider tips, and more! If you like what you see, you can receive free and daily updates via email or RSS.
Print This Post Print This Post

How and Why You Should be Using Google Webmaster Tools – Part 2

December 15th, 2008 | | Crawlability

Last week I wrote part 1of the how and why into Google webmaster tools as a tool for improving your organic rankings and traffic.  Just to reiterate, Google webmaster tools is an account you can set up for free that gives you viable information regarding your website and how Google crawls your site.  From setting up your WMT account to checking your settings and reviewing the diagnostics page these tools help you fix error pages on your site and help you understand how often and which pages Google indexes.  So as part 2 of Webmaster tools, today I’ll go into detail on reports such as statistics, links, sitemaps and tools and how they too can help increase your organic rankings and traffic.


In the statistics section of Google WMT there are several reports you can view and analysis to help improve your search engine rankings and traffic.

Top Search Queries: Like that of Google analytics, WMT can deliver the top search queries.  The information in the report includes the percentage of searches for a particular keyword, the keyword itself and the position organically in which the keyword is ranked via Google.

What Googlebot Sees: the what googlebot sees section is a report that shows all of the anchor text located within your site. Anchor text meaning the text you have that links to either page to your site or another site altogether.  Since the Googlebot reads this anchor text, it’s important that you don’t have any general anchor text like, “click here”.  It’s important to include keywords in your anchor text in order for the Googlebot to read it and help your rankings improve.  A good example of great anchor text is using a keyword like, view our “natural dog food today” with a link going to your natural dog food page.

Crawl Stats: The crawl stats report shows you the Googlebot activity within the past 90 days.  There are graphs that show the number of pages crawled per day, number of kilobytes downloaded per day and the time spent downloading a page.  The number of pages crawled each day is the more important metric you should be looking at.  Depending on the total number of pages in your site, depends on whether your average number of pages crawled is high or low.  If you have a high maximum number of pages crawled in one day and your average is much lower than that – you may want to work on updating more of your sub pages more often.  Meaning pages that are harder to get to, that are located deeper within your site need to be updated with new, fresh content more often.  When you do update new content to these pages, be sure to update your sitemap.

The second report in this section is ‘your pages in google’.  This is a chart detailing the pagerank of your pages within Google on a high, medium, low or not yet assigned level.  On the screenshot below, I have many pages on my site that have a low level of pagerank, which means I need to update the content on more pages of my site and incorporate keywords within that content in order to improve my pagerank.

Index Stats:  Index stats lists the different searches you can perform in Google to actually see the pages that Google has information on.  For example, if you wanted to see all the pages that Googlebot has indexed on your site, you can type in the google search box site:www.yoursite.com, and hit search. This command will list all of the pages in your site only that Google has indexed and that users can access through the Google search.  There are many other commands you can type in the google search to get information about your site and here in the index stats Google has listed a few of them for you already with brief descriptions.

Subscriber Stats: this report is only used for sites that have a feed that users can subscribe to, like a blog.  To learn how to implement feeds via WMT you can read about it in the Google help section.


There are three reports within the links section of WMT.  Pages with external links, pages with internal links and site links.

Pages with external links: This report gives you a list of pages that have links pointing to your site from other sites.  If you c lick on the number under external links you will see all the pages that are currently linking to your site and the data Google indexed those pages.

Pages with internal links: Otherwise known as cross linking, this report shows the pages within your site that have other pages in your site linking to them.

Site links:  Site links are additional links to your site that Google will sometimes generate in order to help users navigate your site.  Unfortunately, Google only generates these sitelinks occasionally and it’s not something you can request or do yourself.  The only way to get sitelinks from google is to keep working with your site and improving the content and navigation. Below is an example of a search result with sitelinks:


Sitemaps are one of the first steps you should implement when beginning SEO for a client. Sitemaps tell Google about the pages on your site they may skip over.  Once you’ve submitted your sitemap, (should be named ‘sitemap.xml’) you can come here and see the total urls in the site map and how many of those Google has indexed.  You can also find error information on your sitemap here and the last time Google downloaded your sitemap.  To submit your sitemap via WMT all you have to do is type in the name of your sitemap in the open field box next to your domain name, and hit submit sitemap. It’s that easy!


Analyze robots.txt: This tool I have found to be pretty valuable.  The robots.txt file that you submit to your site tells Google not to index these particular pages. When you use the tool in WMT it will tell you when the last time google downloaded your robots.txt file and what the status was.

The tool also allows you to check certain pages of your site to see if they have been blocked accurately or not by your robots.txt file.  You can add a page from your site in the check and click check at the bottom of the page to get your results.

Generate robots.txt: If you don’t know how to format your robots file it’s pretty simple. But here Google will help you. All you need to do is add the files or directories from your site and Google will add them to your robots.txt file. Once it’s complete, you can download it right from this page and upload it to your site.

Manage site verification: this tool allows you to view the verification file that is associated with your Google WMT account. This file should be found in your website so that WMT can pull all this information from your site.  This w ill will also tell you whether or not any other verified owners were found for your site.

Remove URL’s: The remove url’s tool allows you to submit certain url’s that you don’t want Google to crawl. However we recommend using the robots.txt file instead of this feature.

Enhance 404 pages: We all know that 404 pages are the dreaded pages that we get when url returns a page not found error.  Google knows this can be frustrating to users and has created this page to help webmasters create custom 404 pages that will help direct users to other pages within your site they might find useful.

You can either use the code Google provides on this page and add it to your generic 404 page, or if you are familiar with CSS, you can use this guide to change the appearance of the 404 widget on your custom 404 page.

Gadgets: Gadgets is a tool that allows you to access Google WMT form your own homepage to get errors, stats and details about your site faster than logging into Google WMT.

Well there you have it. Again, all of these reports and tools in Google Webmaster tools will help you understand how Google is crawling and indexing your site.  By making frequent changes and updates to your site the better your chances are of achieving higher rankings and traffic.  Use these tools listed in this post and take steps to provide your users a better user experience when they enter your site.  Good luck!

Facebook   IN   Stumble Upon   Twitter   Sphinndo some of that social network stuff.