Your Own Computer Can Spider the Web, Thanks to MyLiveSearch
Oh boy, with the buzz MyLiveSsearch is getting today, expect to see the company snapped up by Google, Yahoo or Microsoft in the next few months. Why?
MyLiveSearch claims to have finally figured out how to tap into the “hidden web” – all the stuff that is too fresh or too deeply stored for other search engines to discover.
It works through a small browser plug-in. The search terms are put through Google, or other indexed search databases, but those results are treated as “starting points” alongside the user’s bookmarks and other popular web hubs.
From there, the live search takes over, crawling through hundreds of web pages connected to those starting points in search of more information relevant to the search.
My first question was, can indexing the live web scale? But then I realized, it doesn’t have to. MyLiveSearch is not looking to index all of the hidden web, just the parts that an individual user may have an interest in. With faster computers and fatter broadband connections, MyLiveSearch is entering the space at just the right time – you couldn’t have done this just a few years ago.
While Google wouldn’t comment on reports that it has met twice with MyLiveSearch creators, Rob and Mark Gabriel, that didn’t stop Rob Gabriel from adding fuel to the fire.
“This technology could be snapped onto any of the major search engines and improve them.”
If it does exactly what is says on the can, then you should fully expect one of the big engines to come knocking.