Four Things You Should Know About Collecting User Reviews

Nice write up Greg, the impact of customer reviews seem to be a hotter topic of late. I’m surprised that the reviews don’t directly drive organic traffic, but I would think there are ancillary benefits. In addition to Jaan’s note, I would think the regualar addition of content (via reviews) to individual product pages would be beneficial to the ranking of the page itself.

Greg,

Three questions…

1) Did overall sales rise for them? Or did the same number of sales simply rebalance existing volume toward 4 and 5 star products?

2) Did they require a minimum number of reviews for a rating to appear? I’d hate to have one or two reviews move sales dramatically, esp when the 3 star reviews have a significant negative effect.

3) given that, did they feel that a 5 step scale was working or are the considering a half star increment, etc.

4) and finally, how did they deal with issues of gaming the reviews? It must be tempting to spike your sales by putting a five star review up or conversely to hurt competitors by giving them a 2 star review. I’m not commenting on the ethics of this, but if these results are typical and they become widely known it’s going to be very tempting for people to game reviews.

Greg,

thanks for the reply… during the dotcom era the company I was with bought a consumer review site and they also subscribed to the quantity overcomes gaming view… they just didn’t show scores until a product had five reviews…. they did other things, but that and watching IPs worked pretty well.

Nice posts the last couple of weeks btw… thanks for diving into this issue.

Maybe I missed this but, was there a difference in new customer sales vs. repeat customer sales with the rating system?

I wouldn’t have expected the decrease in sales for the three star reviews, at least not a 10% decrease. It’s interesting that a three star review which is essentially a non-positive and non-negative vote would have more effect than the positive four star review.

It makes me wonder how a 10 star review would be received. Would the increase in sales begin with a rating of six or would it take an eight to generate an increase.

On the other end would a three star system work better with both two and three ratings seeing an increase in sales and only a one star review lead to a decrease.

I’d think there would be an ideal rating division to get the most sales increases.

I would like to point out that a 3-star average review doesn’t necessarily mean a non-positive or non-negative vote. In many instances it means the users either loved the product or hated it.

Here’s a short study discussing this topic on our web site:
http://www.summize.com/review-fatigue.html