Robots (.txt) in Your Sitemaps Report
February 12th, 2006
Google Sitemaps is now reporting Googlebot's view of robots.txt file at your site. It would be useful if you are using it to restrict Googlebot from crawling some sections of your website. The report will assist you to debug your robots.txt file, to show if it's working the way you want it to be. Log in to your Google Sitemaps and have a look now.
If my article helped you solved your problem, consider buy me a beer!
Share this article: del.icio.us | digg it
Related posts:
