Dopo aver risposto, giustamente, a Rupert Murdoch con un RTFM! nei confronti della sua polemica per l’indicizzazione delle notizie da parte di Google News, che ricordiamo essere già in causa con l’Italia per abuso di posizione dominante, Google presenta un nuovo user agent per Google News in modo da evitare possibile cause e consentire tramite robots.txt la limitazione d’accesso.
Notare anche l’ironia dell’excerpt:
Today we are announcing a new user agent for robots.txt called Googlebot-News that gives publishers even more control over their content. In case you haven’t heard of robots.txt, it’s a web-wide standard that has been in use since 1994 and which has support from all major search engines and well-behaved “robots” that process the web. When a search engine checks whether it has permission to crawl and index a web page, the “check if we’re allowed to crawl this page” mechanism is robots.txt.
Popularity: 1%
