SEO weather reports – the trackers that track a number of keywords in the search results, and report on the volatility in those searches – have become a pretty popular way for webmasters and SEOs to kind of gauge what is happening in the search results on a daily basis.

The topic came up about how Google evaluates changes and algo tests they are running in the search results, then moved over to the SEO weather reports.  John Mueller talked about how all the tests Google runs can easily get caught by rank checking bots.

Here was what John Mueller said specifically about the weather reporting SEO bots in the last Google Webmaster Office Hours.

I imagine these SEO weather tools, they recognize these kind of changes from time to time as well.   The tricky part is pretty much everyone when they’re searching, they are in at least a number of experiments.

There are always changes happening in the search results, there’s always things we are testing for, to kind of see are these things good changes or bad changes, if we change the UI slightly and add a couple pixels here, how to people respond to that.  That’s pretty much always happening.

A lot of times when I see the SEO weather reports “oh, it’s like record high weather today”, when we ask around, it’s like “well, we’re not doing anything crazy, it’s just a bunch of tests running at the same time maybe, and maybe this testing tool is running into queries where they do see those changes.

So it’s not necessarily that something has actually changed but we are always making changes, we are always tweaking and testing and seeing what we need to improve.

There are many times that one tracker might show significant changes while the others show normal fluctuations.   And other times they show major changes when Google confirms no major algorithm changes were made.  But with all searchers – and all the various rank tracking bots – being in any number of SEO experiments at any time, it can be hard to know when to trust those weather reports and when not to.

But there is definitely some confusion over SEO weather bots that track serps, since there are some people who thought these bots somehow bypass any experiments.  But since they are essentially scraping the search results – which is against Google’s TOS – they have to mimic a typical user, which means they do see all kinds of random experiements, as we have seen with Dr. Pete Myers posting screenshots seen via Mozcast from these very experiments.

Most SEOs tend to take the weather reports with a grain of salt, and then check their own rankings to see if fluctuations in their own search results.

The topic also came up last year, where John also talked about the inaccuracies of ranking weather reports.