Marketing Pilgrim's "Content Marketing" Channel

Sponsor Marketing Pilgrim's "Content Marketing" Channel today! Get in front of some of the most influential readers in the Internet and social media marketing industry. Contact us today!

Paid Content: To Index or Not to Index, That is the Question


Hello, you there! Person reading this post. How did you find it? If you’re a loyal MarketingPilgrim follower then maybe you came in here via our daily email update. Perhaps you came in off a social media link or you clicked a bookmark. If you’ve never been here before, then chances are you came in via a search engine. Welcome. We’re happy to have you and we hope you come back often.

Search engines are vital to the success of any site with written content. So why would you want to stop them from indexing your site?

The UK newspaper The Times stopped them because they felt it ran counter to their paid content business model. Back in 2010, the newspaper blocked search engine spiders as part of their conversion to digital subscriptions. Apparently, they believed that their good name alone would drive traffic to the site.

They don’t believe that anymore.

According to PaidContent, The Times is about to open the doors — a little — allowing search engines to pull the first few lines of every article in hopes of snagging new subscribers. Wow, radical thinking. Before they exterminated the spiders, The Times said that up to 40% of their traffic came from search engines. Why in the world would they want to cut that off?

I suppose The Times thought that exclusivity would outweigh the benefits of random searchers. Now, they’re saying they need to give indexing a try as a marketing experiment.

Most of the online papers that offer digital subscriptions allow readers to view several free articles a month and a portion of articles beyond that. It’s the difference between watching half a movie than being asked to pay to see the ending, or just seeing the title before you have to pay. Hmmm. . .could be a new concept for video on demand. .  .

I can’t see how The Times ever justified cutting off the search engines in the first place. I don’t care how large and how loyal your customer base is, you have to constantly bring in new customers or you’ll fail in a few years.

Digital subscriptions are still an experiment. They work for some and not for others. Tablets are making them more popular as slowly, magazines are cutting down their printed pages in favor of digital ones. The trouble is, there are still hundreds of excellent, free options for digital news so paid content models need all the help they can get and that includes a hand from Bing, Yahoo and Google.