Google Webmaster Tools Adds Robots.txt Update Tool

If you have ever made a mistake with your robots.txt file – either by letting Googlebot crawl something you didn’t want it to or because you accidentally blocked Googlebot (or someone hacked your site and did it) – it can be some nail biting times until you see Googlebot picking up the new robots.txt changes.

Google has added an addition to their robots.txt tool.  Now, you can do the usual check of your robots.txt file, but then you can also verify the live version and then submit it to notify Google that your robots.txt file has changed and request that Googlebot crawl it again.

You can access the new tool within Google Webmaster Tools by selecting “Crawl” in the left sidebar, and then choosing robots.txt Tester.

The new change is now live in Google Webmaster Tools accounts.

The following two tabs change content below.

• Bio
• Latest Posts

Jennifer Slegg

Founder & Editor at The SEM Post

Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.

Latest posts by Jennifer Slegg (see all)

• 2022 Update for Google Quality Rater Guidelines – Big YMYL Updates - August 1, 2022
• Google Quality Rater Guidelines: The Low Quality 2021 Update - October 19, 2021
• Rethinking Affiliate Sites With Google’s Product Review Update - April 23, 2021
• New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met - October 16, 2020
• Google Updates Experiment Statistics for Quality Raters - October 6, 2020