The shady side of software reviews

When searching for the next software to buy — and the next provider to buy from — a common source of information is software reviews. They let us know what the service is like and give us an idea about how the solution helps other people like us.

Or do they? The fact is, software reviews have a shady side. Indeed, they shouldn’t be the be-all and end-all when searching for a new software solution.

So, what is the shady side of software reviews?

Casting shade on reliability

Software reviews carry uneven reliability. To be reliable, it needs to be replicable. Translated into reviews, this means the more positive reviews, the more reliable the positive sentiment. In theory.  

When you rely on reliability, the number of reviews becomes paramount. For example, a negative review is going to hurt the average for a small start-up developer with only a few reviews so far. But that same negative review would scarcely have an impact at all for a large company with thousands of reviews.

In short, the more reviews you have, the less weight they carry. And this puts smaller software companies at a disadvantage. Not to mention, reliability doesn’t even mean validity. That is, the reviews aren’t always true.

Shady validity and fake reviews

So, where are invalid reviews coming from? For a start, the customer isn’t always right, and reviews only give one side of the story. Further, the motivation behind the review isn’t always clear. It could come from genuine delight and balanced experiences. But it could also come from shadier motivations.

For example, with review sites, there’s a major issue with companies buying software reviews. That is, purchasing fake positive reviews is a common shady practice. In more extreme cases, some businesses have had negative reviews bought for them, as a form of attack on the competition.

In either case, the reviewer hasn’t tried or used the software in question, and the reviews are nothing more than hot air.

Encouraging false positivity

Other shady practices include ‘incentivised’ customer reviews.

These are the software reviews that companies directly ask customers for, in return for a reward. For example, the customer gets a discount on their next purchase, a thank you gift, or free postage, etcetera, for leaving a review.

Of course, software enterprises pick and choose who they solicit reviews from. So, they’ll favour customers that will give them positive reviews. But there is a difference in experience between organic reviews fuelled by delight and an encouraged review.

Cast in shadow

As a result of these shady software reviews, it’s near impossible to separate the authentic from the disingenuous.

And even if fake reviews act as a leg up to generate organic opinions, they’re still affecting the average reliability of the reviews. They still negate the weight that real opinions have.

For instance, a given solution could have 100 positive reviews and 10 negative ones. That means only 10% of people had an issue. That’s not so bad. Now, consider that 50 of those positive reviews were, in fact, fake. That means that 20% of real customers had an issue with the software — a much more concerning percentage.

Favouring big enterprises

All this cumulates into reviews often favouring larger enterprises.

Businesses with larger budgets can throw money at generating a host of great fake reviews. They can also afford to promise big rewards for customers who leave a review. And they can afford to thrust their solution into the spotlight with paid ‘featured product’ positions. (Showcasing their fake software reviews.)

All this disingenuous activity serves to generate increased attention for the big-budget brand. More customers are drawn to the spotlighted and heavily reviewed products. After all, more reviews, theoretically, means more reliability. Then, these genuine customers mingle their own (real) reviews with the paid ones. This further widens the gap between the bigger enterprises and their smaller competitors.

So, the large enterprise gets larger, and their tactics are imperceptible to the consumer. Smaller competitors, meanwhile, get lost in the money-driven review site model.

Navigating the shadow

It is important to bear this in mind when reading reviews and take them with a pinch of salt.

Instead of completely trusting enterprise review sites, look for a balance of positive and negative. Then, weigh the pros and cons you find for yourself. Be wary of excessive positive or excessive negative reviews.

It’s also advisable to visit the companies providing the enterprise applications you are considering. From there, you should be able to take advantage of a free trial. This allows you to assess suitability and quality for yourself.

In other words, when it comes to software reviews, it’s important to review the review.

Useful links

Product feedback: the proof of the pudding is in the eating

The tech market and why the winner takes it all

A buyer’s guide to evaluating SaaS products