GoogleBot Understands Comments

By Jeremy Luebke on August 27, 2007

GoogleBot Understands Comments

Blog comments have long been a link building tool for many SEOs (unfortunately). Before the nofollow tag, comment abuse ran rampant, and now with the nofollow tag in place, many people search high and low for blogs that have it removed.

I am personally against the nofollow tag. It’s one of the most absurd ideas to come out of the search engines in a long time. It has done nothing to stop the daily influx of automated comment spam it sought to stop.

That being said, I am also not one to remove the nofollow attribute from blog comments. All that accomplishes is attracting more attention from spammers. Next thing you know you’re on a few public yesfollow spam lists and probably hundreds of private ones. I can think of a few ways to turn a yesfollow blog into a good link building scheme that in the end, only benefits you. Luckily there are easier and safer ways to build link equity.

NoFollow Isn’t Needed. Google Understands Comments

So all of this leads me to a point. Google doesn’t actually need nofollow to weed out blog comments. They know good and well exactly what part of a blog post is content and what part is a comment.

Case in point. A few weeks ago I received a Google alert on my name. It was a blog alert and had 2 links in it.

Jeremy Luebke commenting on How SEOmoz Built One Million Links in …
By Jeremy Luebke
Congrats. That carpel tunnel syndrome you will soon have from blogging so much all seems worth it now

Comment on Top10 in Google by Jeremy Luebke
As someone who sells links I have been thinking this for a long time. I know of no commercial serp that isn’t littered with sites buying links. For most companies you either buy them or you don’t rank. If all those sites got nailed? …

Notice how Google links directly to the comment anchor tag and not just the page itself. Naylor runs an out of the box blog package, or so it seems (even though this link is dead now). But SEOmoz is running a custom blog program which I doubt Google went out of their way to profile. This means that Google is pretty smart when it comes to detecting what is a comment and what isn’t.

So then why did Google push for the nofollow? I don’t have the definitive answers but it does help support Michael Gray’s theories on Google wanting the nofollow for more than just comments before it was actually released because they obviously don’t need it to identify blog comment links.

With this evidence, I also have to wonder if people targeting yesfollow blogs are also wasting their breath. I would actually hope so, but even if that is the truth, it unfortunately won’t stop the spammers from doing what they do.

Similar Stories in: General, Search News | Forward: Email This Post

22 comments on “GoogleBot Understands Comments”

FREE SONY PS3 LOL (not spam) Says:

August 28th, 2007 at 5:24 am

Why are you against nofollow? What an entirely silly position to take.

If you are against the idea that links should be used as part of the organization (and thus search topology) of the web, then I will accept your statement, and curiously ask how you would replace back-link-based search.

If you like the googleithm (google+algorithm), why could you possibly dislike nofollow?

You own a domain. wtfyouwant.com

This can haz flavor. I mean. This has pagerank.

You now wield that pagerank and are able to VOTE on other sites relevance and importance by molding the site and its links. (as well as writing and emphasis around the link + sqrt(-1)^pi/i^pi(sqrt(pi^i)) )=0.840622423 + 0.541621586 i) or whatever google does.

Now – up until this point, do you agree that your site has pagerank, pagerank is a useful search tool (i.e. pagerank driven search engines) and links are important in that mixture?

At the next election, would you like others to share how your vote is cast? Using your details to make votes for people you entirely do not like?

Yes? ok you win.

No? The shock, really? Well, not using nofollow on any part of your domain where users are able to instigate links (comments, forums, profile homepage links (see profile spam)) you are letting others abuse your reputation (pagerank).

Hopefully google will drop pagerank for spammed up sites, and let clean, unspammed sites retain theirs.

Now, if you agreed with me up until now, I propose the only reason you do not like nofollow is that you are entirely ignorant (which I find hard to believe given the content of this site) of the purpose of nofollow, how google works, and what a link means – as well as the relationship between site and user content.

Which part do you disagree with?
Bushido Says:

August 28th, 2007 at 6:46 am

I think the best way to deal with spam is creating a governmental organism that penalizes (with a fine or maybe even prison time) companies that hire spam services.
Maybe it’s true that spammers are every day harder to locate, then follow that spam link and bust the company that hired the unethical service.

That’s my 2 cents.
William Says:

August 28th, 2007 at 10:19 am

Hey, GoogleBot. I’m a comment. Nice to meet you! Do you understand me? If so could you please do your job a little bit faster?
Mark Barrera Says:

August 28th, 2007 at 12:06 pm

I have held the same theory as you that Google might have the ability to determine what part of a page’s copy is user contributed comments.

I am still looking for proof that they are actually treating comments any different from the rest of a page.

In your Naylor example, it looks like Googlebot was crawling the Comments RSS feed and saw your name and notified you. Does this mean that they actually know that it is a comment and should be treated differently or is it just another feed to them that they need to index? I don’t think this one presents enough proof of treating the comment differently.

In the SEOmoz example, they don’t have a comment feed, but they do have permalinks for each individual commment. This is the one that really supports your belief because the title of the link that you got from your alert is not given in a feed or anywhere else like it was in the Naylor example. Do you think Google has done a manual job on this one?
Jeremy Luebke Says:

August 28th, 2007 at 12:36 pm

@Mark

This is not the first example I have seen of these. They have been happening for a long time, I just finally got around to blogging about it.

I definitely do not think they took the time to include SEOmoz as a special case. That wouldn’t make any sense. I see what you mean about the Naylor title. I doubt Naylor is pinging Google blog search with it’s comment feed though, so that means that Google went out of the way to read the comment feed if that is the case.

The phenomenon seems restricted to blog search where it is actually useful. There is not much real use for linking to anchor tags in normal search. That doesn’t mean it’s not used in the main algo though.

As for whether they are treating comments differently than content, I’m not sure. But it seems that they definitely have the ability to.

@FREE SONY PS3 LOL (not spam)

The only thing silly here is calling someone else’s position silly anonymously with that sort of name. Did you even bother to read the entire article and comprehend what the actual topic was about?
Mark Barrera Says:

August 28th, 2007 at 12:41 pm

Jeremy – Thanks for your thoughts. I always enjoy reading your articles as they tend to go a bit deeper than what everyone else is posting.
David Eaves Says:

August 28th, 2007 at 5:12 pm

I know of some sites that are ranking pretty well off do follow blog comments, I know this one guy who has gone around calling himself SEO something and targeting do follow blogs, he has got a very good rank on Google from it. Google needs to put more weight on high quality paid directories, ones that have editorial integrity.
Sante Says:

August 29th, 2007 at 5:16 am

Hi,
I have to disagree with you about NOFOLLOW attracting more spammers. I removed the NOFOLLOW about 3 months ago, maybe more. There hasn’t been a spike in spam, it’s business as usual as far as my blog is concerned.

I’ll admit that there are people who try to get thru with silly comments like “great post !” but it’s a minority.

People aren’t all like me or you and won’t go out of their way to leave a comment just for the sake of leaving one – without the NOFOLLOW your blog answers the fundamental question:

WHAT’S IN IT FOR ME?

and your answer is – A LINK

So it works both ways, you get a comment as a token of appreciation of your work as a blogger, and they get a quality link – that’s the way it works in a business, you give and you get …
Andy Beard Says:

August 29th, 2007 at 6:28 am

SEOmoz does have a comments feed – it is a blogwide implementation though, all or nothing which you can become overwhelmed with in a matter of hours, if not days.

That spammer has been touting that junk PDF in loads of forums, but the spammers are extremely easy to spot because they don’t realise that the community is fairly self policing, and subscribe to comments.
When you see 3 or 4 comments from the same person appear in your email box in a matter of minutes, you know someone is working down a list.

I am sure we will hit 400 members in the community on Bumpzee soon, more than twice the number as is included on the spammer list (who you linked to without nofollow?).
David Airey Says:

August 29th, 2007 at 9:44 am

I was about to mention that you linked to the spam list without nofollow.

What I find funny is that people are installing DoFollow Plugins and asking to be included on the PDF list, like it will bring them some kind of fame.
Nhu-Chi Says:

August 29th, 2007 at 7:19 pm

According to a panelist at the SES Conference, Google’s push for nofollow blogs is a ploy, their real intentions for nofollow is to regulate text link ads. So your argument is very valid. Google is more than capable of separating spam and real content.
It's Really This Easy To Add A Link To Your Comment - Dawud Miracle @ dmiracle.com - Says:

August 31st, 2007 at 11:20 am

[...] Also, one further note. Some blogware will not let you add HTML to your comments. So try it and if you get a message that HTML is not accepted, you’ll know. Trying it won’t hurt anything. [...]
Perde Says:

September 13th, 2007 at 5:23 pm

I think there is nothing wrong with giving Google juice to commentors. Besides, nofollow does not stop automated messages as was said so nofollow for blogs is not a good idea.
Perde Says:

September 13th, 2007 at 5:29 pm

I highly doubt if Google understands comments. Google can’t even detect hidden keywords, which is one of the oldest form of spamming, and in some cases send visitors to spammed page just because some 1px hidden keyword is there. Everyone is talking about improved algorithm but still some guys spam with the same methods used a decade back and get away with it.
Arthur mcbeth Says:

September 18th, 2007 at 12:59 am

I like to use Googlebot in the tag. I believe the bot likes to see its name.

meta name=”googlebot” content=”index, follow
stor perde Says:

November 18th, 2007 at 6:09 am

I’ll admit that there are people who try to get thru with silly comments like “great post !” but it’s a minority
NinjaTickets--Comparison shop event tickets Says:

December 15th, 2007 at 1:39 am

Great Post!
NinjaPenisEnlargement Says:

May 13th, 2008 at 4:32 am

Great post!
NinjaClitorisEnlargement Says:

May 13th, 2008 at 4:33 am

Perceptive post!
kiralama Says:

July 15th, 2008 at 5:46 am

very nice but perceptive post!
Manny Says:

July 18th, 2008 at 10:07 am

Very funny all the people spamming this, lol may be you should have the nofollow on?
Ilker Karaagac Says:

May 29th, 2009 at 5:55 pm

http://www.garantitatil.com
ssuming the deal is around $20m, Rev/employee and company value/employee are identical to GOOG. Avid gets $2m in earnings on $5m in revs and a good property for $20m…not bad.

Marketing Pilgrim