If you have ever made a mistake with your robots.txt file – either by letting Googlebot crawl something you didn’t want it to or because you accidentally blocked Googlebot (or someone hacked your site and did it) – it can be some nail biting times until you see Googlebot picking up the new robots.txt changes.
Google has added an addition to their robots.txt tool. Now, you can do the usual check of your robots.txt file, but then you can also verify the live version and then submit it to notify Google that your robots.txt file has changed and request that Googlebot crawl it again.
You can access the new tool within Google Webmaster Tools by selecting “Crawl” in the left sidebar, and then choosing robots.txt Tester.
The new change is now live in Google Webmaster Tools accounts.
[…] Google Webmaster Tools Adds Robots.txt Update Tool, The SEM Post […]