Posted November 23, 2005 12:12 pm by with 0 comments

Tweet about this on TwitterShare on LinkedInShare on Google+Share on FacebookBuffer this page

So, on Monday, apparently, body snatchers stole Brett Tabke. Brett disallowed all spiders to crawl due to rogue spiders crawling 12 million pages of WebmasterWorld during the week of the WebmasterWorld search conference. Brett also says he spends 5-8 hours a week battling these rogue bots.

Search Engine Watch is reporting that WebmasterWorld is out of both Google and MSN Search results.

A robots.txt file disallowing all spiders + Two Days = Out of Google and MSN. That’s it.

Now, Danny says that there may be some manual factors at play here with Google, sleuthily pointing out “Interestingly, checking the Google Directory — which is powered by the Open Directory — there is no listing for WebmasterWorld in the same exact category as you’ll see at the Open Directory. It suggests that the robots.txt ban had the effect of pulling WebmasterWorld not only out of the Google web search results but Google Directory listings as well. That would be an entirely new thing I don’t recall hearing happening before.