Posted August 9, 2007 10:30 am by with 3 comments

Tweet about this on TwitterShare on LinkedInShare on Google+Share on FacebookBuffer this page

Over at TechCrunch, Michael Arrington, picks up on a concern over Google News, raised by TechMeme’s Gabe Rivera.

One thing that bugs me: they?re now hosting original news content, yet they prohibit other aggregators from crawling it (per robots.txt restrictions and TOS). Of course Google News relies on the openness of other organizations with original news content.

image The bottom line? Google’s terms prohibit anyone from spidering or manually copying any content from the Google News service. This wasn’t a big deal when Google News was simply aggregating news from elsewhere, but it’s somewhat hypocritical now that Google News is allowing comments–and hence is now a content host themselves.

Arrington hopes this is just an oversight on Google’s part–and I do too. It’s ironic that Google fights many battles to protect its right to crawl your news content, while at the same time preventing access to its own content.

  • Google rarely plays by the same set of rules they set out for everyone else. If you get banned from AdSense they ask you to send in as much information and server logs etc that you can. They in turn won’t even tell you which site to investigate. They ask people to identify paid links and report spam but in return they don’t even allow us to see supplemental results anymore.

    I’m obviously a bit biased but reading this I was left with the thought of “That seems about right…”

  • With the bankroll they throw at the lawyers these days, I am sure that it will be a while before anyone can break down these walls that Google has built around its content.

  • I wonder if Google’s boast would wither if every single person didn’t log in to Google for a single day…

    Just a thought.