Those ubiquitous star ratings were supposed to help us separate the good from the better and best. But 90% of Yelp reviews are 5-star, 50% for Amazon’s products, and even then, the average star rating was 4.2. In short, they fail at separating the good from the bad. For marketers, those most interested in pushing our buttons in one direction or another, star ratings are poor predictors of a product’s performance. The current study finds that while star ratings are quantifiable, the words accompanying those ratings are more predictive.
A word about words
Words often convey more than their definition. Words can carry emotional and energetic tones – think of it as the harmonics that accompany musical notes. Consider, as the researchers suggest, enjoyable and impeccable – both are positive in describing a product, but doesn’t enjoyable have more emotional echoes? Similarly, which you prefer your spouse to be exciting or lovable? Both positive, both with emotional overtones, but I think exciting sounds more energized, lovable a little more laid-back.
The scientists involved in this study used a system that categorizes the positivity, emotionality, and energy of words. They applied this classification to the comments that accompanied those star ratings for movies, books, TV commercials, and restaurant reservations (pre-COVID).
Findings
- For movies, the researchers looked at the first 30 reviews for all films on Metacritic.com, an aggregator of public opinion, from 2005 to 2018. The researchers based their judgment of the movies on box-office revenue. 81% of movies were rated above average on the site. Those star ratings could identify the bad but could not separate good, better, and best. The emotionality of the comments is far better at that task.
- For books, they turned to Amazon’s reviews from 1995 to 2015; on-site sales were the researcher’s metric. 91% of books were rated above average. Higher star ratings were, in some cases, a negative predictor of book sales. Again “greater emotionality was predictive of more book purchases in 93% of genres.”
- For commercials, the researchers looked at the 2016 and 2017 Super Bowl advertising focusing on the tweets about those commercials, roughly 200,000 for the 94 commercials. The new Facebook followers garnered by the businesses in the following two weeks were the outcome. They used the USA Today rankings of the commercials as their “star” ratings. The pattern continued. The tweets' emotionality was far more predictive of the new Facebook followers than the USA Today rankings.
- For restaurant reservations, the researchers utilized the first 30 Yelp reviews for two months in 2017. Table reservations came from OpenTable.com, an online reservation service. 92% of Yelp reviews were above average. Star ratings were predictive of online reservations, but star ratings no longer had any predictive value when emotionality was added to the model.
The Take-Away
When considering the wisdom of the crowd, it seems we often follow the adage that if we have nothing nice to say, we simply do not speak. How else could we explain the large number of reviews that were better than average? It also seems that stars, as a summary metric, may identify the bad and the ugly but fails us when it comes to the good. Even then, word choice matters. Energized, emotional word choices move the needle more.
“… attitudes based more on emotion tend to be stronger and more consistent across contexts and time. One reason for these outcomes is that these attitudes tend to be stored more strongly in memory. Stronger links in memory predict what individuals think about and what captures their attention in their environment, thereby providing a general guide for behaviour.”
It is interesting to speculate whether emotionality holds up in the sphere of political discourse, specifically in public health policy's politicization. Could it be that conspiracies and mistruth gain the most traction not from repetition but from the words they choose?
Source: Mass-scale emotionality reveals human behaviour and marketplace success Nature Human Behavior DOI: 10.1038/s41562-021-01098-5
Chuck Dinerstein, MD, MBA
Director of Medicine
Dr. Charles Dinerstein, M.D., MBA, FACS is Director of Medicine at the American Council on Science and Health. He has over 25 years of experience as a vascular surgeon.
