Wine, Blind Taste Tests, and Your Money

You’re standing at the grocery store looking at a large selection of wines. You know you want a red wine to go well with your pasta meal this evening. One is $5.99 per bottle. Another is $19.99 per bottle and has a little tag bragging about the score it received in some wine magazine. Which one is likely to bring you more enjoyment?

Believe it or not, the difference between the wines amounts to a complete coin flip. Professional wine judges, when faced with duplicate blind taste tests, fail to accurately repeat their grading of wines. From this study, with my own emphasis added:

Wine judge performance at a major wine competition has been analyzed from 2005 to 2008 using replicate samples. Each panel of four expert judges received a flight of 30 wines imbedded with triplicate samples poured from the same bottle. Between 65 and 70 judges were tested each year. About 10 percent of the judges were able to replicate their score within a single medal group. Another 10 percent, on occasion, scored the same wine Bronze to Gold. Judges tend to be more consistent in what they don’t like than what they do. An analysis of variance covering every panel over the study period indicates only about half of the panels presented awards based solely on wine quality.

These results – 10% of the tasters repeating their grading – are within the realm of ordinary luck.

In fact, there has really only been one thing that clearly “moves the needle” when it comes to wine tasting results: the price on the wine bottle. From this 2008 study:

Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less. For individuals with wine training, however, we find indications of a positive relationship between price and enjoyment. Our results are robust to the inclusion of individual fixed effects, and are not driven by outliers: when omitting the top and bottom deciles of the price distribution, our qualitative results are strengthened, and the statistical significance is improved further. Our results indicate that both the prices of wines and wine recommendations by experts may be poor guides for non-expert wine consumers.

Yep, if anything, people like expensive wines less than cheaper wines if they don’t know the price, but once they do, those more expensive wines suddenly taste a whole lot better!

So, when you’re facing that wall of wines, don’t be swayed by the price tag nor by the wine scores or the wine notes. Unless you have a truly rare palate, it’s not going to make a whole lot of difference to you. In fact, from both my reading and my personal experience, the biggest impact on how the wine tastes is the food that goes with it and the company that shares it with you.

Now, here’s the swerve: the phenomenon of price not directly matching quality doesn’t stop at wines.

Think of all of the instances you can where something with a higher price is perceived to have a higher quality than something with a lower price. It’s pretty easy. Tablet computers. Cheeses. Beers. Cell phones. You can list thousands of these kinds of examples.

Now, given this wine study, how much of that perceived quality is simply due to the price tag and not due to the inherent quality of the item?

It’s an interesting question and it very quickly becomes a hard thing to answer. You simply can’t trust the idea that “more expensive equals better.”

So, what can you trust?

First, divorce yourself from the idea that “expensive is better.” There are so many factors that go into pricing – marketing is a huge one – that the actual quality of the product may or may not be reflected in the price. Work on unlinking the two in your mind.

For me, one powerful way of doing this was blind taste tests. My friends have done a number of these over the years and I’ve liked the low priced or moderately priced item just as often as the expensive item. Try it yourself! A blind taste test can be a very interesting way to spend an evening with friends, especially when you all get a good laugh out of realizing that the most expensive item probably wasn’t the best one.

Second, look for quantifiable information about the products. By this, I mean specific numerical facts, like those you might find on a Nutrition Facts label. Don’t rely on number-based ratings – review scores generally don’t mean too much unless you trust the reviewer (more on that in a minute).

Pure numbers themselves don’t tell you the whole truth about a product, but they do give you something real upon which to base comparisons. I constantly use Nutrition Facts labels to compare food items, for example. They’re really useful in helping to identify lower calorie and lower salt foods.

Third, don’t be afraid to try out the cheap option. If it’s cheap, after all, you’re not out too much if you end up buying something that’s not all that great. It’s a lot easier to tolerate a bad bottle of wine if it cost $5 than if it cost $20, for example.

We buy a lot of generics and store brands. We buy cheap wines, too. Sometimes, these don’t turn out well. Often, however, they’re quite good and the choice leaves money in our pocket (where it belongs). There’s no need to toss a fist full of money at an unknown item, especially when you have no real idea how well you’ll like it.

Fourth, use your own judgment. Take advantage of any and all situations in your life where you might be able to compare options without spending money and without seeing the prices.

One great example of this is using a friend’s purchase to sample things. I’ve tried many different items in the homes of my friends and they’ve tried items at my house. I’ll often ask whether that particular item is any good and, almost always, they’ll let me try it if it’s at all reasonable to do so.

Finally, use reviews from trusted sources. For larger purchases, you do need more information than a side-by-side numbers comparison can give you. In those situations, rely on reviews from sources that you trust.

Why is trust so important in a review? People who write and record reviews are motivated by something when writing that review. Ideally, you want reviews from people who are motivated to give an unbiased review. Unfortunately, that’s not always the motivation of people. Some reviews come from people who have unrelated axes to grind, such as people who drop one star reviews on items because they didn’t like the shipping. Some reviews come from people who are paid to give glowing reviews, or from people who are just trashing a competitor’s product. Some reviews come from people talk about off-label use for a product, using it for things that it was never intended for. Some reviews come from people never bother to read the manual and figure out how to use something.

If you don’t know whether or not any of those conditions are true, then you should take that review with less than a grain of salt. Instead, look for trusted sources from organizations and individuals who have built a reputation for solid reviewing of products. For example, I trust Consumer Reports for most of the products I buy, and I have a handful of people whose opinions I trust within my hobby areas of interest.

Of course, that means not relying on random reviews on Amazon to tell you what is “good” and what is “bad.” You can occasionally glean some information from these reviews if there are hundreds of them and the criticisms are very consistent, but one or two one star reviews are essentially useless as they can come from individuals with unclear motivations and unclear levels of honesty.

The purpose of all of these tactics is to completely divorce yourself from directly connecting the price to the quality of an item. As with wines, the sticker price can often lead you astray and actually prevent you from making a correct judgment when it comes to the quality of the item and how useful it can be in your life.

If you take away nothing else from this post, it’s this: a higher price often doesn’t equal higher quality. Sometimes, it just creates an illusion of quality where you convince yourself it must be good because of the higher prices. Avoid that illusion of quality and keep your cash in your wallet where it belongs.