The SEM Post

Latest News About SEO, SEM, PPC & Search Engines

You are here: Home / Google / How Many URLs & Duplicates Google Knows About

How Many URLs & Duplicates Google Knows About

at PST By

google urls duplicatesDuring the State of Search keynote yesterday in Dallas, Texas, Gary Illyes shared a pretty incredible stat – Google knows about 120 trillion URLs, and of those URLs, 60% of them are duplicate.

Now keep in mind Illyes does not say these are all indexed and ranking – merely that 60% of the URLs Google knows about are duplicate.  But this is still an incredible number – more than half of them are duplicate!

We see Google regularly kick duplicates out of the search results – completely if it is a case of hardcore spam – or at least to not show them until someone clicks on the link at the bottom of the search results to reveal duplicates.  There are also duplicates that get removed via a DMCA filed with Google, to remove stolen content from the search results.

Google doesn’t tend to release crawl stats very often, so it is nice to hear that there are 140 trillion URLs that Google knows about, as well as the fact 60% of those known URLs are duplicate.

The following two tabs change content below.
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.

Sign up for our newsletter


Comments

  1. I wonder if they also classify unintentional duplicates, like URLs with utm params, as part of that 60%.