X
    Categories: AppleSEO

Apple Officially Acknowledges Their Applebot Web Crawler

Apple seems to have finally officially acknowledged their “Applebot” webcrawler with a new webpage added to their support site this week. Apple is using this crawler for both Siri and Spotlight Suggestions, among other products.

User-agent strings will contain “Applebot” together with additional agent information. For example:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)

What is really interesting about this announcement is that not only are they following robots instructions but if those robots instructions do not include Applebot specifically but do mention Googlebot, they will follow the instructions for Googlebot.

We actually reported on this crawler back in November, as people start to see showing up in logs. And while Apple did not comment publicly about the crawler at the time, it was determined that it wasn’t a search engine that Apple was planning but rather their crawler to improve Siri. However it was not crawling under the official name of “Applebot” at the time, however people were able to identify is because of the IP address used.

If course the million dollar question is whether or not they will be using any of the data gained from Applebot to launch their own search engine in the future.

The following two tabs change content below.

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.
Jennifer Slegg :Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.