The SEM Post

Latest News About SEO, SEM, PPC & Search Engines

You are here: Home / Google / Google Updates Quality Rater Guidelines for YMYL & Offensive Results

Google Updates Quality Rater Guidelines for YMYL & Offensive Results

at PST By

Google has made another update to the Google Quality Rater Guidelines, coming only a couple of months after the last release, which saw major changes made to target fake news, science denail and clickbait, among other things.

This updates sees Google bring added clarification to specific areas of the guidelines and other minor changes.  The biggest changes are to news sites with Your Money Your Life, as well as further clarifications as to when a page or site should get the Upsetting-Offensive flag, which was introduced with the last update to specifically target fake news as well as other sites publishing fake information.

Here’s the full breakdown of what changed.

General Guidelines Overview

This section is primarily for the raters themselves about how ratings are done.

Releasing Tasks

This was removed from the beginning of the rater guidelines, but Google expanded this section and included it in the final section, which is detailed below.

Page Quality Rating Guideline

This section primarily talks about overall page quality, and there was one significant change made for news sites with regards to YMYL.

Your Money or Your Life (YMYL) Pages

Google makes a point of stating that not all news articles are considered to be YMYL pages.  Previously, they did not make it clear that not all news would fall under that category.

In the “News articles or public/official information pages important for having an informed citizenry”, Google added “Keep in mind that not all news articles are necessarily considered YMYL.”

So this means that some types of news sites will not be held up to the much higher standard of YMYL, which was not clear in the last version.  This is good because some types of news sites really don’t impact Your Money or Your Life, yet they would also be held to this higher standard.

To clarify, news sites that would somehow potentially impact the future happiness, health, or financial stability of users ARE still being held to YMYL.  If your news site does not fall under that, it would not.

Needs Met Rating Guideline

This section has a variety of changes made, primarily to Fails to Meet and porn queries.

Examples of Fails to Meet (FailsM) Result Blocks

This is an interesting addition.  Google has added an additional example of an inaccurate featured snippet. This shows they want their raters to be aware that featured snippets are not always right, and that raters need to consider that and rate them with Fails to Meet appropriately.

Here is the added example, which is a much more obvious featured snippet than the other example used in the guidelines:

Needs Met Rating for Possible Porn Intent Queries

Google has made some changes to this section.  Most notably, they removed additional keywords as types of searches that could have possible porn intent but also non porn intent.  The removed keywords are girls, gay, thong, spanking.

They also make it clear that they want raters to rate based on the query being non-porn in intent.  Here they also changed their example keyword from bikini to breast.

Here is what the new section looks like:

Some queries have both non-porn and porn interpretations. For example, the following English (US) queries have both a non-porn and an erotic or porn interpretation: [breast], [sex]. We will call these queries “possible porn intent” queries.

For “possible porn intent” queries, please rate as if the non-porn interpretation were dominant, even though some or many users may be looking for porn. For example, please rate the English (US) query [breast] assuming a dominant health or anatomy information intent.

For the examples in this section, they removed the “pictures of girls” example.

Needs Met Rating for Clear Porn Intent Queries

They removed a bit in here.  The italicized part was removed:

For very clear porn queries where no other intent is possible, assign a rating to the porn landing page based on how helpful it is for the user. Even though there is porn intent, the page should still be assigned a Porn flag.

Not clear why specifically this was removed.

Reporting Illegal Images

Here, Google has changed multiple references of “reporting to your vendor” to “reporting to your employer/company” throughout this section.

This is likely in response to the fact raters must now be employees of the companies doing the ratings, and not mere contractors, a change Google began requiring for the companies that are responsible for hiring those to do the ratings.

Using the Upsetting-Offensive Tag

Google has brought some additional clarification to how raters should interpret if a result deserves the upsetting-offensive tag or not.  Google has added this section:

Important: Please think about the purpose of the page and how the content is presented when determining whether to assign the Upsetting-Offensive flag. There are many results around the web that focus on sensitive topics such as child abuse, violence, or racism; however, you should assign the Upsetting-Offensive flag based on the purpose, type, and/or presentation of the content on the page —not because the topic itself is sensitive or potentially upsetting.

For example, a result with content that encourages or graphically depicts child abuse should be flagged as Upsetting-Offensive . However, an accurate informational page about  child abuse (such as child abuse statistics, prevention, etc.) should not be flagged, even though child abuse itself is a sensitive topic that users may find upsetting.

This serves more of a reminder about how simply because a topic is a sensitive one, does not mean it automatically deserves the upsetting-offensive tag, but rather it should be based upon factors such as page purpose.  For example, even though reading statistics about child support is upsetting, it is still quality information and should not be flagged as upsetting-offensive.  On the other hand, a page on how to abuse a child would be considered upsetting-offensive.

This is very new to the quality rater guidelines and was brought up into response for queries like “Did the Holocaust happen” as well as the increase in fake news sites and articles.  But the clarification seems to imply that perhaps some raters were having trouble distinguishing if a result really should have the upsetting-offensive tag, or if some raters were flagging because the general topic was upsetting to them.

Needs Met Rating for Upsetting-Offensive Tolerant Queries

Again, more clarification has been brought to a page which is considered upsetting-offensive and the needs met scale.  Google is reminding that even if the needs of the query was met, it doesn’t make an upsetting-offensive queries less upsetting-offensive.

Remember that you should assign the Upsetting-Offensive flag to all upsetting/offensive results, even if they satisfy the user intent.

For example, someone could search for “prove the Holocaust is fake”, and even if the query was technically met with a page that IS upsetting-offensive, doesn’t mean it loses that designation simply because the query matched the result.  Remember, Google is trying to prevent these types of results from surfacing for various queries.

Needs Met Task Page Screenshot

Curiously, Google has removed this screenshot from the guide.  This screenshot was one of the few we have that gives insight into the interface the raters use when it comes to rating search results and webpages.

Using the Evaluation Platform

This section is specific to the raters doing the actual evaluations, but there is often some interesting information found within this section for SEOs and site owners.

Simplified Needs Met Tasks

The big change here is Google has increased the size of the screenshots showing the backend – they were so tiny before they were impossible to make out.

They also changed the wording which also includes a new reference to “contextual results”.

Some Needs Met rating tasks may not require ratings for all blocks. In these blocks, the Needs Met and EAT rating scales will be greyed out and annotated with the text “No Rating Required”.

The greyed-out results (sometimes referred to as “contextual results”) are shown to help you understand the query and inform your ratings. You can click on them when you feel like they will be helpful in determining your ratings.

It is not clear why Google felt the need to refer to these as contextual results however.

Understanding the User Location on the Task Page

Google has updating the wording with regards to the maps feature that sometimes sees a map included with some results, so searchers can evaluate if the query and results match the location.

While most query-based rating tasks have a user location listed below the query, a map will often be provided as well. This map can help clarify where the user was physically located when the query was issued in a more precise way.

Notes about Using the Needs Met Rating Interface

This section was completely removed, but it served the actual raters from a technical point of view.

Pre-Identified Duplicates

They included guidelines for raters about how the system determines duplicates, and removed some clarifying points:

● Results annotated by “ Dupe of… ” may be given different ratings/flags/comments.
● Results annotated by “ Same as… ” may not be given different ratings/flags/comments. Their Needs Met and EAT ratings, flags, and comments will be automatically transferred to each other.
● You cannot uncheck dupes that have been automatically detected and preidentified.

Releasing Tasks

As noted above, this section was removed from the start of the guidelines. Google also added a new screenshot showing the reasons for released tasks.  Of note is the new “I did not sign up for adult content” addition.

More Notes about Using the Needs Met Rating Interface

Most of this section has new parts, but they are directly related to the backend technical stuff for the raters.  Anything that could impact SEOs in this section has been included above.

Final Thoughts

This update is not nearly as big as the last one published earlier this year, which saw significant changes made to fake news, science denial and clickbait.  This particular update seems to be more targeted to the raters workflow with some more detailed clarifications where there was previously some questions about the ratings.  If this is your first time reading the guidelines, my previous update has many more changes within it.

The full guidelines have been published by Google here.

The following two tabs change content below.
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.

Sign up for our newsletter