"Have nothing in your houses that you do not know to be useful or believe to be beautiful." - William Morris

Please Rate This

Posted: January 11th, 2019 | Author: | Filed under: Interaction Design, Product Management | Tags: , , , , | Comments Off on Please Rate This

The other day I met the founder of a rapidly growing startup for lunch. The first thing we talked about though wasn’t strictly business, it was about where we had met to eat. The venue was a popular restaurant in the centre of London that had been chosen by someone else as part of organising his busy day in London. Both of us were unsure we wanted to eat there, as London has an amazing selection of high quality restaurants and this was in both our opinions not one of them, but since he had to catch a flight we decided to stick with the plan to make the most of our time.

Spoiler alert: the food was solid and neither of us got food poisoning, so this is not an early Halloween food horror story.

The restaurant chosen is actually very highly rated on TripAdvisor, in the top 150 of the near 19,000 London restaurants they have on their site. This seems like a good reason to choose it, at least on the face of it you’re not going to go far wrong eating there. That doesn’t mean that it’s a restaurant that everyone will enjoy in the same way. In my experience of using TripAdvisor there will be the obvious bias towards tourists given their target market, and within that a strong American focus depending on the city. This leads to a natural skew in what types of restaurants are preferred, which hotels are favoured and so on. Personally I do use TripAdvisor regularly when I travel, but tend to use bad ratings as an exclusion criteria, then the reviews as a positive decision input. The upshot is – if you want a more interesting local restaurant recommendation you may need to dig around for a more local site. Or ask that obsessive foodie friend you have as they are guaranteed to have a list of amazing places to go around the world (thanks Engin).

Of course on every site and service we use these days we have ratings. That AirBnB you want to book? That Amazon purchase you want to make? That Uber you just took? Over 1,000 people rated it an average of five stars, go for it! We generally believe people expect a rating, it gives potential customers a sense that other people have been there and trodden this path before us giving a sense of security. The rating itself may not even be as important as the number of people who’ve given it – 4.5 / 5 stars seems good, but less so if only a few people have rated it.

Ratings are not the only mechanism – there are also likes, or “would you recommend this product to a friend?” questions with a yes / no answer. Simple, binary and blunt. Personally I prefer that approach, as it is quick, emotional and talks strongly to the concept that we only recommend something based on our own experiences. “I thought that was the best food I’d ever had”, oh – did I mention I’m a strict vegan?

Which comes to the crux of the problem – does that rating or recommendation that someone else gave apply to the person currently using it to make their own decision. When your company is small with few users this is often not a problem, you attract a focussed group of early adopters with likely similar needs, but as you grow the problem changes. That’s why it’s fundamentally important to constantly review core behaviours in your product even if they’ve seemed to work for a long time – people change, even if your product doesn’t and what was acceptable to them even last year may not be now. At some point moving to a ‘people like you, liked this’ approach looks to have the most merit. It leads to different challenges, how do you know which people are like each other? How do you deal with anonymous users?

Netflix has gone for this different approach, they offer a match percentage based on what you’ve watched before. After you watch a show or series you can give it a thumbs up or down, and as you watch more – in theory – the match and recommendations for you get better and better. This allows them to push recommendations across their huge content library and keep you binging on more shows, a core KPI. Their recommendation approach has taken a long time to develop and is not cheap to run, they have around 1,000 people working on their recommendations that get updated every 24 hours. Understanding what content works best for everyone on their system is a core differentiator to their business model so this investment makes total sense.

Every approach has its own merits, and the approach you take will need to vary over time. Likely the only truth is that if you use no rating mechanism at all you are missing out on a wealth of value both for you and your users. In every market there are subtleties, and as product people we have to understand both the different dynamic of our product offering, and also the expected normal behaviours of our target market. Even a small mis-match that sends a customer to a choice they don’t ultimately enjoy can damage their trust in us, and trust is the most valuable commodity.