The SEM Post

Latest News About SEO, SEM, PPC & Search Engines

You are here: Home / Google / Updated Google Quality Rater Guidelines: International Search Results & Conspiracy Theories

Updated Google Quality Rater Guidelines: International Search Results & Conspiracy Theories

at PST By

Google has once again very quietly updated their Google Quality Rater Guidelines with a brand new version.   This update comes as the third update this year so far (following ones in March and May), only months after we saw major changes with fake news.

There are some notable changes, especially for those SEOs who work in international markets, as this update seems primarily focused with international non-English results.  This is particularly where there are non-English languages being searched in, as well as cases where there might be both English and non-English mixed results from Google.

We also see a new conspiracy theory example, which seems to have been added to give more clarity to the raters.  Since Google is trying to remove fake news, conspiracy theories and the like from the search results, this gives raters a better example of what Google sees as a conspiracy theory.  In a previous update to the guidelines earlier this year, they were change to reflect Google’s desire for things like fake news, inaccurate content (ie. dubious medical or scientific claims) and conspiracy theories to rank lower in the search results, and the most recent for Your Money Your Life and offensive results.

And we see a removal that has to do with ads, which is likely because so many ads are behavior targeted that using the type of ads on a site – such as porn ads on a non-porn site – that could have had nothing to do with the site being rated, but rather due to the type of sites the rater visits, or due to malware.

Here is a break down of all the things added, removed and changed in the latest version of the Google Quality Rater Guidelines.

7.0 Lowest Quality Pages

Here they make a slight change, which seem more grammatical in difference.  Under the types of pages that are considered to be the lowest quality pages on the web, they removed the itallicized part below:

Pages or websites which are created to make money with little to no attempt to help users.

7.10 Examples of Lowest Quality Pages

They have added a new example, which is pretty interesting in light of all the fake news and conspiracy theory additions made this year, as this one is a specific example of a conspiracy theory page – although it is definitely one of the less plausible of the range of conspiracy theories they could have chosen.

Lowest: Deceptive page purpose – unsubstantiated conspiracy theories.

Google wants the raters to look specifically at sites or pages where the conspiracy theories are presented as fact.  “Presents unsubstantiated conspiracy theories as if the information were factual.”

And raters should rate them: “Lowest quality MC: demonstrably inaccurate content.”

The example continues to show that raters should be able to find credible sources that have debunked the conspiracy theory.

So this doesn’t seem to be targeting conspiracy theory pages specifically, as there are sites about conspiracy theories that actually debunk the conspiracy theory.  But rather this is targeting sites that are presenting the conspiracy theory as fact.

It is great that Google is including specific examples of conspiracy theories to give guidance to the raters.  This was definitely one of the grey areas previously.

Page Quality Rating Guidelines

Introduction to rating page quality tasks

The changes here only applied to the raters for doing tasks.

Understanding Mobile User Needs

Locale and User Location

Google continues to do some clarification to locale type queries, including this section which was renamed to “Locale and User Location” from the original “Task Location (Locale) and User Location”.

They added specifically that anything related to user location is referred to as “Locale” in the guidelines.

All queries have a task language and task location (referred to in rating tasks as the “Locale”).

Special Content Result Block Examples

They updated the Coldplay screenshot to show what videos on mobile in the search results currently look like.

Needs Met Rating Guideline

Examples of Fails to Meet (FailsM) Result Blocks

Just a side note that the stairs featured snippet example has been updated in the search results, so the live search results no longer show the erroneous featured snippet.

Needs Met Rating for Porn Results

Some interesting changes were made here that relate to porn content that appears on both porn sites and non-porn sites.

Needs Met Rating for Clear Non-Porn Intent Queries

Google has removed the section about porn ads being displayed on non-porn sites.  The only possible reason I can see for them removing this section is that it could be influenced by such things as a quality rater having malware on their computer, or possibly because the ads are being targeted to the rater based on previous porn site visits.  And there is always the chance of an ad network throwing porn or close to porn ads into the ad network rotation.

While seeing porn ads on a non-porn site is never a good thing, it is good to see that Google is preventing sites from being flagged by raters due to the raters own behavior.  And as for porn ads via nefarious activity from an ad network, there definitely have been cases where some ad networks let porn ads “slip through” (although the debate is whether they allowed them knowingly or not), but there is onus on the site owner to chose ad networks with good reputations for ad quality.

Here is the removed section:

Sometimes, the MC of a landing page is helpful for the query, but the page happens to display porn ads or porn links outside the MC, which can be very distracting and potentially provide a poor user experience. The query and the helpfulness of the MC have to be balanced with the user experience of the page. Use your judgment and represent users in your locale.

Foreign Language Flag

Notable changes to this section.

Using the Foreign Language Flag

In the first section of bullet points, the last two bullet points in this section swapped places.

A rather interesting change was made to the foreign language example.  Google removed the Ukraine / Russia example that was previously in this section and changed it to a Catalan example.  It isn’t clear why they made this change, other than it changed the example to being one of 3 possible language choices.

The new example:

For example, most Catalan-speaking users in Spain also speak Spanish. Therefore, for rating tasks in Catalan (ES), the Foreign Language flag should NOT be assigned to landing pages in Catalan, Spanish, or English.

In the second set of bullet points in this section, some more notable changes were made.

First, they tightened up the reasons for using the foreign flag when there are locale issues in play.

They changed “Please remember to flag all foreign pages with the Foreign Language flag, even if the query “asks” for a foreign
language page” to “Please remember to flag all foreign pages with the Foreign Language flag, even if most users in your locale
would expect or want a foreign language page for the query.”

From the last bullet point, they also removed “You may look at MC, SC, Ads, and even the website the page is on” which was used by raters to determine language for a page.  In light of the previous ad removal for porn sites, it does make one wonder if this was removed due to behavior targeted ads.

All Needs Met Examples

For some reason, Google has swapped the positioning of the bars showing the EAT and Needs Met for the examples with screenshots in this entire section.  It is unclear why this was done, unless it is to better match how it is presented in the tool the raters use for evaluating.

Needs Met Rating for Foreign Language Results

Again, tightened up on locale issues.

They changed “If the query is clearly “asking” for a foreign language result, then the Needs Met rating of the foreign language page
should not be FailsM” to “If the query clearly indicates that most users would expect or want a foreign language result, then the Needs Met rating of the foreign language page should not be FailsM”

Rating English Language Results in Non-English Locales

This section also saw some changes, mostly to better emphasize the issue with English in non-English locales, which primarily involves unexpected English results when one would expect all the results to be in the language for the location searched from.

The following section has changed, the part in italics was added.

However, rating can be more difficult when the query includes English names, words, etc., or when it’s unclear whether English results would be satisfying for a particular query. Please use your judgment and knowledge of your locale to determine the appropriate rating.

They also added a very important paragraph to this with more details on English versus non-English results. Again this seems to be targeting international raters who are rating in either a single non-English language, or rating (or being familiar with) multiple languages.

Important: Please keep in mind that every locale will have unique considerations regarding the number and variety of  languages (such as official languages, regional languages, local dialects, etc.), writing systems, and keyboard input languages commonly in use. While this guideline may not include examples for your locale, it is important that you represent users in your task location and culture in order to interpret the query and rate results. When in doubt, please assume that users would prefer results in the task language unless the query clearly indicates otherwise.

Examples of English (and Non-English) Results in Non-English Locales

This is an entirely new section, although the examples were simply moved from the previous section.  Again, Google is wanting to provide clearer definition for when English results appear when another language is expected.

They have added the following section, although a small part of it was rewritten from the previous section in the old version of the guidelines.

The section includes some examples using Hindi (IN) and Korean (KR) as the locales. In both cases, we cannot assume that users in these locales—i.e., Hindi-speaking users in India, or Korean-speaking users in Korea—are able to read English. Unless most users in the locale would be satisfied by English results for the query, we will consider them unhelpful or even useless ( FailsM ).

Some of the examples have been reordered for cohesiveness.

They have added a new section with examples as well.  They provided more commentary about non-English results that are being rated. Here is what was added:

On the other hand, there may be queries where the needs of most users would actually be satisfied by English language results, even if the query itself is typed in the task language.

For example, for queries about global businesses and organizations, users may expect or want to visit the English language version of the business/organization’s official website in some locales. Similarly, for queries seeking technical information such as manufacturer part numbers, product specs, scientific or chemical formulas, etc., the answer to the query may be typically expressed in the English language in some locales.

For these queries, users may expect or want to see English results in order to satisfy their need. Please use your judgment and knowledge of your locale to determine the appropriate rating.

Overall

There were some significant changes, but mostly as it pertains to non-English results.  So it could be that Google is working on quality and spam in non-English language results.

The conspiracy theory change isn’t that unexpected, other than it wasn’t added to the guidelines originally with all the fake news tackling additions.

You can download the latest version of the Quality Rater Guidelines here (July 27, 2017).

The following two tabs change content below.
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.

Sign up for our newsletter


Comments