User-written reviews: trustable?

After reading yet another story of a vendor in our space trying to stack the deck, I am reminded of my opinion that user-written reviews must be looked at with a very large grain of salt.  Click Read More to find out who the guilty party is this time.

This time it’s none other than Carbonite, the providers of an Internet-based backup service that competes with EMC’s Mozy.  It offers a similar service as Mozy at the same price ($50 annual).  According to a story in The New York Times, which sources this original blog post by someone calling themselves “. You’ll see that Carbonite’s CEO (Mr. Friend) read and responded to the comments to this article.  Since one of the biggest complaints in those comments (written prior to his comment) was that Carbonite was submitting fake reviews in Amazon, it’s hard to believe that he didn’t already know this was a problem.  So the VP of Marketing was apparently doing it and the CEO seems to have known about it, but they didn’t do anything about it until a full story about it came out in the New York Times.

Belkin seems to have done the same thing.  According to this story on The Register, Belkin was paying people to submit positive reviews of their products.  Interestingly enough, they were using Amazon’s Mechanical Turk service to find and pay people to submit positive reviews on Amazon.  As to who knew what and when, another story in Real Tech News, some of the fake reviews come from Rudy Magna, a Senior National Account Manager at Belkin.  It’s hard to claim that creating these product reviews was an “unofficial policy” (Belkin’s response to the first story) when a “Senior National Account Manager” is doing them.

I’ve seen this kind of nonsense in comments to blogs and responses to questions in the Backup Central Forums.  I’ve also seen companies posting negative reviews about their competitor’s products.   Some vendors use what’s called Sock Puppetry, where they sign on as one user that asks a question, then sign on as another user with an answer to that question.  A few years ago, I witnessed employees from a CDP vendor sitting in the audience of a panel discussion about CDP.  They asked leading questions to their representative on the panel and tough questions to everyone else. Michael Rowan busted them from the stage and they were asked to sit down and shut up.  It was hilarious.

Here are my thoughts on this.  First, it’s just wrong.  It’s called being a shill or a plant.  According to a line in the Wikipedia article, it is even illegal in some jurisdictions.  “It may be considered a form of unjust enrichment or unfair competition, as in California’s Business & Professions Code § 17200, which prohibits “unfair or fraudulent business act[s] or practice[s] and unfair, deceptive, untrue or misleading advertising“.”   I see people comment on the Carbonite story with comments about how the product works.  That isn’t the point.  The point is that what they were apparently doing is dishonest and possibly illegal.  Besides, who would believe a user-submitted positive review as a comment on a story about a company caught submitting their own user-submitted positive reviews?

Second, shilling is far more common than you think it is.  If the makers of a $50 year backup service and a $8.44 mouse pad are doing this, what do you think companies are selling hardware and software costing hundreds of thousands of dollars are doing?  Are some of them doing it?  How many are doing it?  I have no idea.  But even one of them doing it severely taints the value of all user-generated content if it hasn’t been validated by a 3rd party company (like the guys over at TechValidate).

I truly am incensed by this (because it’s just wrong), but it would be duplicitous of me if I didn’t say that the business idea that I’m working on involves helping companies evaluate new technology and incidents like this one will be cited among the reasons that I’m creating such a company.  I believe that opinions of a given product’s customers should be included when evaluating a product — if you can validate that they are real opinions of real people that aren’t bought and paid for.

Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

5 comments
  • Quick side note regarding the $8.44 mouse pad from Amazon. Total after shipping comes to $14.02! Are they joking! It is a mouse pad! The shills could post that this is the holy grail of mouse pads and I still wouldn’t pay $8.44 to get one let alone $14.02. But according to its reviews others did. In fact one reviwer even commented “They are great for the price” and another “It’s just a basic mouse pad, but the price is excellent!!!” Great for the price? Yeah, if it was free.

  • What’s this world coming to. Who can you trust these days? Maybe there is something in the stimulus bill for help with these kind of cheaters like there is for banking.

  • I am part of a startup sales.

    When you see someone asking for a solution similar to what you have, Its always tempting to write comments 🙂

    But, after a few mistakes (some of them on Backup Central Forums), I always add to signature who I am.

    Jaspreet