The SEM Post

Latest News About SEO, SEM, PPC & Search Engines

You are here: Home / Google / New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met

New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met

at PST By

Google has released a brand-new Google quality rater guidelines, coming close to a year after the last one was released in December 2019. This update doesn’t have some of the sweeping updates and changes that we’ve seen in some of the previous updates. There are many interesting things Google has added, stressed, or changed, so SEOs should still be paying attention to these changes.

But one of the most popular changes in these guidelines will be the addition of a change log for the guidelines. While the change log isn’t completely thorough, it does serve as a heads up for what has changed. And they’ve also included in the change log all the changes made for the past two years.

Google has put more emphasis on Needs Met this time, and as I said at Pubcon yesterday, Needs Met is an area that SEOs really should be paying closer attention to because it is the Needs Met score that Google seems to be really working on with their algos.

Google hasn’t made any changes to E-A-T in this update, although it did receive an extensive update previously. So the fact that there were no changes made to E-A-T, nor specifically to YMYL, but this new emphasis on Needs Met should be a sign for SEOs to consider when working on their SEO strategy as well as their landing page maintenance and refreshers.

So let’s get started with all the changes in the new guidelines.

Table of Contents

So there are changes to the table of contents, but they are all reflected into the later sections of the guidelines. But I did want to make a note of them.

0.1   The Purpose of Search Quality Rating

Google is mainly added more clarity to the guidelines about quality raters not being able to directly impact the search results. This is probably the most common question I get about the rater guidelines, so it’s nice to see Google add the section in, especially for those non-raters to read the guidelines.

Here is what they added:

Your ratings will not directly affect how a particular webpage, website, or result appears in Google Search, nor will they cause specific webpages, websites, or results to move up or down on the search results page. Instead, your ratings will be used to measure how well search engine algorithms are performing for a broad range of searches.

This section was definitely long-overdue, and it is great to see that Google has finally added this, as it is something that is tangible that we can show people about what the raters can and cannot do.

But to add my own thoughts here, it’s important to remember that just because quality raters can’t directly impact how well your site is doing in the search results, if quality raters are consistently ranking your site poorly, or sites similar in quality to yours, then you need to be aware that Google is actively working on algorithms to make it so that your type of site won’t be ranking as well in the search results.

So, just because these raters can’t directly cause your site to lose rankings in the Google search results, it doesn’t mean that your site is golden.  Google wants to make what raters are seen as high quality to rank highest, and the sites and pages they rate lowest to not rank well at all in the search results.  And they are working on algorithms to do just that.

0.2   Raters Must Represent People in their Rating Locale

This section received an unusual update by taking out one small phrase, but that has much larger connotations with how raters handle some more sensitive topics, especially those who have viewpoints that are considered “normal” in their eyes, but to the general public might not be so much.

Google has removed the two words “the norm” from the second paragraph of the sentence.

Here is how it originally appeared. The new version simply has “the norm” removed.

Unless your rating task indicates otherwise, your ratings should be based on the instructions and examples given in these guidelines. Ratings should not be based on your personal opinions, preferences, religious beliefs, or political views.  Always use your best judgment and represent the cultural standards and norms of your rating locale.

Now, this can be taken obviously in multiple different ways. But the only reason I think this could possibly up and meet with because some raters are perhaps holding this up as an example that maybe some of their personal offensive behavior could have been influencing their ratings simply based on the following to the letter “norms of your rating locale.”

So while this is a seemingly tiny and minor change, it has larger implications so it is clear why Google removed this.

0.6 The Role of Examples in these Guidelines

This is the brand-new section that Google has added for this version of the guidelines.

Google has done some changes to some of the examples used in this version of the quality rater guidelines. However, if you’re very familiar with the guidelines, the majority of the examples are the same from the last version, except for primarily some of the new sections that Google added.

Parts of this section have been taken from the second half of the 1.0 section in the last set of guidelines. Google has added a new introductory paragraph to the section that was heavily revamped from the old one.

The old version:

The examples in these guidelines are very important. Please view each one and keep in mind the following notes:

The new version:

The examples throughout these guidelines are very important to illustrate how the concepts defined  in each section should be applied in rating tasks. Please view each example and keep the following in mind:

The three bullet points that follow this were identical to the old version.

Also, a note on the examples, you can click through on the majority of the links to examples to see a clearer version of the screenshot for each example. While earlier versions of the guidelines with given error when you try to view them, the last few years the larger versions of most of the screenshots if you want to see something more clearly in them.

7.4.2 Malicious Pages

The malicious page section has received a bit of an update, specifically regarding ensuring that the raters are not clicking through to malicious results that could end up infecting their computers.

If you do click on a link and encounter a malware warning from your browser or antivirus software (example), please do not continue to that page. You can assume that the page is malicious for the purpose of rating and use Lowest.

So obviously situations where a site may be infected through something a third-party has placed, like when a website gets hacked, it should be noted that raters are still supposed to give those pages or sites a lowest rating.  Raters are not differentiating between your standard hacked WordPress site versus a deliberately malicious site.

Google is also asking raters to differentiate between warnings for things like malware versus other types of browser warnings such as related to certificates or content filter warnings.

Note that not all warnings your browser may display are related to malware (e.g., other types of browser warnings include certificate acceptance requests, content filtering warnings, etc.). If you are unsure or hesitant to continue to the page for any reason, you can release the task.

12.8.2 Special Content Result Block Examples

Google made a few smaller changes here.

First, Google removed this:

and may not require additional “clicks” or page loads.

And in it’s place, they added a new sentence just to make what they are trying to convey a bit clearer for raters.

Some SCRBs may also have links to landing pages.

Google then also remove this entire paragraph:

Because mobile phones can be difficult to use, SCRBs can help mobile phone users accomplish their tasks very quickly, especially for certain Know Simple , Visit-in-Person , and Do queries.

This removal simply seems more because people are a lot more comfortable with mobile phones than they were when this was originally added multiple years ago.  Since that time it was introduced, mobile search share has also seen an increase to a point where over 50% of all searches are on mobile devices.

Part 3: Needs Met Rating Guideline

This is one section where we have seen some more clarification and promotion of Needs Met.  I feel it’s important for SEOs to focus more time on the Needs Met aspect of the Quality Rater Guidelines and how they can integrate it better into their landing pages, by ensuring that they are meeting the needs of the users.

13.1 Rating Result Blocks: Block Content and Landing Pages

Google reworded the section about Special Content Result Blocks (SCRBs) and how they relate to landing pages these SCRBs are linked to.

Here is how the second paragraph in the “What to rate” part of the table originally appeared:

In addition, think about whether a user would click on the link(s) if available, in order to satisfy their user need.

And how it appears now:

Some Special Content Result Blocks may have links to landing pages. In these cases, think about whether a user would click on the link in order to satisfy their user need.

This change, to focus a little bit more on the links to the landing pages and how well they meet the user’s needs in special content results boxes ,should indicate that SEOs should be paying attention to the content that appears in things such as featured snippets.  They should ensure that the content in featured snippets, as well as the content that’s on the landing pages, are resulting in a high Needs Met. We know that Google likes to highest quality sites for the featured snippets, so if you have any featured snippets, you should ensure that you are meeting the needs of the users if they click through, as well as for the snippet Google’s featuring.

Quality raters are also looking at whether someone would need to click the link or not. And the raters aren’t given any guidelines to whether one way is better than the other, such as a SCRB needing a click through or not. And there are a lot of ways that SEOs are trying to manage their featured snippets in a way that still encourages people to click through while still containing enough of the relevant information in a featured snippet able quote that Google would promote it as the featured snippet.

From Google’s perspective, the things that could be considered SCRBs include featured snippets, local results for business, or any other ways that Google is presenting content searchers, that is not in the traditional” ten blue links.”

Google also added a new notation to the bottom of this section regarding this interaction with SCRBs and the page quality of the landing page that these special results linked to. For some of the queries, raters may or may not be asked to rate the landing page from the page quality perspective, but only as it applies to this section for Needs Met and Quality ratings.

So Google seems to be stressing that not all of these need to have quality ratings for the landing pages, likely because there’s not always that need to click through to the landing page where the information is taken from.

Here is what Google added:

Note: The guidance in this section specifically applies to Needs Met ratings. For Special Content Result Blocks that have landing pages, you may or may not also be asked to provide Page Quality ratings. In these cases, your Page Quality rating should always be based on the landing page. Please refer to Section 14.0 for more about the relationship between Needs Met and Page Quality ratings.

13.6 Fails to Meet (FailsM)

Google made a small change to the section, specifically around raters potentially running into malware or other malicious download pages while doing quality ratings. So this just seems to be more here from a clarification standpoint that raters shouldn’t continue to do any kind of ratings on a webpage or websites that could compromise their computer’s security.  Remember, quality raters do not use ad blockers.

Here is the old bullet point:

Results that harm users (e.g., malicious download pages, “phishing” pages or websites, etc.).

And the new one:

Results that harm users (e.g., malicious download pages, “phishing” pages or websites, etc.). Note that if you encounter a malware warning or untrustworthy site warning from your browser (example) , you can assume for rating purposes that the page is malicious without continuing to the page.

So there might have been some confusion about raters feeling they still needed to continue doing ratings for sites that were compromised or had malicious ads.

14.0 The Relationship between Page Quality and Needs Met

Google changed their guidelines so that this is now appearing in section 14 rather than section 15. Google seems to be stressing a bit more both Needs Met and its relation to quality, so make sense that they prioritize this and moved to the front rather than appearing later in the guidelines after things like Needs Met and offensive results.

Google also made some changes as well.  Google removed the ratings guidance for when a quality rater comes up with a result that either foreign language or did not load.  For some reason, Google had made it so that raters should rate these types of pages as medium on page quality, even when the raters had no way to assess the page.

It had always struck me as odd that’s a page should earn a medium quality rating when a rater couldn’t tell what the page said because it was in a foreign language or when the page just to load in the first place. (Note: Google adds new information to the guidelines later to now say these should not be rated).

Here is what Google removed:

If you are unable to evaluate the Page Quality rating of a Foreign Language or Did Not Load result, you should assign a Medium rating.

Google also made a rather telling change to their examples for this section of the relationship between page quality and Needs Met. In all their explanations for each individual screenshot they have reversed the Needs Met and page quality explanations. So in previous versions, page quality notes appeared first in the explanation, now they’ve swapped all the examples so that Needs Met comes first.

So again, this minor change shows how Google’s emphasizing Needs Met a bit more in this version of the guidelines.

Lastly, Google added a brand-new example to the section, with a featured snippet style content box with images positioned above it relative to the search query.

This example seems to be added in order to illustrate the fact that a search result can have a higher page quality while still failing to meet the query.

15.0 Rating Porn, Foreign Language, Did Not Load, and Upsetting-Offensive Results

Just more notation that this section had swapped places with the previous “The Relationship between Page Quality and Needs Met” section which use to appear as section 15.0.

15.4.2 Needs Met Rating for Foreign Language Results

I commented earlier that I thought it was unusual that Google wanted raters to rate any foreign language results they came across as medium quality, and how it was removed in the previous section. Here, Google has made a change, and the section where they note that  for these foreign-language results, raters should now not assign any kind of quality rating to them and instead leave it as N/A.

Here is what was added specifically:

Note: If you are unable to evaluate the Page Quality rating of a Foreign Language result, you do not need to assign a Page Quality rating and can leave the slider at N /A.

15.5.1 Using the Did Not Load Flag

Minor change here to stress that using did not load is for technical problems on the website and not other technical problems.

Old version:

Did Not Load is used to indicate technical problems that prevent you from viewing any LP content.

New version:

Did Not Load is used to indicate technical problems with the webpage that prevent users from viewing any LP content.

Google has removed some references and has changed it so that raters are now no longer using the do not load flag for landing pages that trigger malware warnings.  They have also added some new specifications about where they should not be using the do not load flag, such as classified listings that are expired, social media links that are no longer available or content that a rater cannot view because it’s behind a subscription or paywalls.

Here is how the part of the section originally appeared in the older guidelines:

Additionally, use the Did Not Load flag for

● Malware warnings, such as “Warning – visiting this web site may harm your computer!”

● Pages that solicit certificate acceptance requests.

And here’s how it appears in the brand-new updated version:

You should not use the Did Not Load flag for:

● Malware warnings, such as “Warning – visiting this site may harm your computer!” (e xample)

● Pages that have removed or expired MC (e.g., expired classified listing, removed social media post, products or services unavailable).

● Pages that are inaccessible because you need a subscription to view the MC.

Google’s also removed the examples of malware warnings.

15.5.2 Needs Met Rating and the Did Not Load Flag

Google added a new notation to the section specific to page quality and pages that do not load.

Here is what was added:

Note: If you are unable to evaluate the Page Quality rating of a D id Not Load result, you do not need to assign a Page Quality rating and can leave the slider at N /A.

Google’s also made some changes here for the examples. First of all, it is broken down the original explanations to note that they are specific to Needs Met. Then Google has added some additional commentary as it relates to page quality under explanations.

Dear updated explanations are both for pages that are for pages with areas, such as 404 errors.  And this reminder that SEOs want to make sure that they are giving visitors to their sites a great experience even if the page is not found.

For the [boy pink snow shows] query, Google has added this under explanation:

Page Quality: This LP is on a well-known, reputable merchant website. Despite the error message, this page does a nice job of explaining the issue and providing a search box, including helpful search tips and customer service information.

For the [bible passages] example:

Page Quality: This page is a well-known, reputable reference website. In addition to links to all passages in the Bible, this error page provides a search box as well as a suggestion explaining how to refine searches.

Google has also added two completely new examples as well.

17.0 Specificity of Queries and Landing Pages

Another section where Google has chosen to emphasize Needs Met over page quality. In other examples in this section, they have swapped positions for Needs Met, and page quality to place Needs Met first.

Again, this is consistent with Google’s emphasis on Needs Met in this update.

24.0 Rating Dictionary and Encyclopedia Results for Different Queries

This is an entirely  new section, which Google is using to highlight searches where the results would be served well with definitions.  It also has information about implied searches and whether searcher is likely looking for a definition or simply looking for more information about the query but not needing a definition per se.

One of their excellent examples is ATM, where most searchers who would be doing a Google search for ATM are likely looking for where a nearby ATM is, and not wanting to know what ATM stands for or generic information about what ATM is. So even though a searcher is not looking specifically for ATM near me, the implied search is that the vast majority of searchers would be looking for nearby ATMs, not a Wikipedia page about automated teller machines.

First, here is what Google says specifically to quality raters coming across dictionary and Wikipedia results.

When assigning Needs Met ratings for dictionary and encyclopedia results, careful attention must be paid to the user intent. Like all results, the helpfulness of dictionary and encyclopedia results depend on the query and user intent.

Dictionary and encyclopedia results may be topically relevant for many searches, but often these results are not helpful for common words that most people in your rating locale already understand. Reserve high Needs Met ratings for dictionary and encyclopedia results when the user intent for the query is likely “what is it” or “what does it mean” and the result is helpful for users seeking that type of information.

Important: Think very carefully about the helpfulness of dictionary and encyclopedia results for ordinary words and common items. If few users would benefit from a dictionary or encyclopedia result for a common word, a S lightly Meets rating may be appropriate. If very few or no users would benefit, then F ails to Meet is appropriate.

Google also includes all these examples for helping readers determine whether a searcher is looking for a definition or whether they’re looking for something that’s an implied query, like the ATM example.

Appendix 2: Guideline Change Log

This is probably one of the changes that SEOs are going to be most excited about. And that is the addition of a change log for what Google has change about the quality raters guidelines.

Obviously, not everything is included for what gets changed, but that can fall into how Google states that there are” minor changes throughout.” And of course, SEOs still want to know all those minor changes too.

For example, if for the December 2019 guidelines update, Google made only two notations of what change, along with these “minor changes throughout” reference, and obviously there a lot more changes than that. But for the October 2020 updates, it is much closer to what we actually saw in this update, so going forward we might see the change log being more detailed than what it shows here for the older updates.

That said, this is a great addition to see and will be pretty useful for SEOs to might not necessarily want to go through the guidelines page vice page themselves, but just get an overview of what changed, and if there is any reason they need to personally read through what the changes were.

Final Thoughts

it was a relatively minor update to the quality rater guidelines when compared to some of the ones we’ve seen recently. I suspect part of that was due to Covid, but I was surprised to not see more of an emphasis on YMYL in light of the pandemic and searchers wanting to get the most accurate information about Covid.  So I did expect to see some more changes or something to health-related YMYL.  The updated guidelines did not make any reference to the pandemic at all.

It wasn’t surprising however, to not see anything specific to the impact of elections, the related search results and the need for accuracy with what candidates may be claiming. But do keep in mind that Google made significant changes after the last US election in 2016 for this exact reason. They introduced many of the checks that were fighting against fake news and having it so that the raters were always ensuring the most accurate news sources were being elevated in the search results and not some of the types of sites that were reporting on conspiracy theories or hoaxes.

The new emphasis on Needs Met is one site owners should be paying more attention to, and was ironically the topic of a conference presentation I did the same day these new guidelines were released.  So if you have been focused primarily on improving E-A-T in relation to the guidelines, you should definitely start looking at Needs Met and how well your landing pages are meeting the needs of the user.

The following two tabs change content below.
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.

Sign up for our newsletter