Wine Tasting: Testing Perception vs. Reality

The other night I had a small get together at my place. The theme of the evening was wine and cheese. I had decided to coordinate a wine tasting, so I gave my guests six different varieties of wine that I wanted them to sample.

However, I also thought this would be a phenomenal chance to collect some interesting data. What possible data could I collect during a wine tasting?

Well, I thought it would be particularly interesting to see if people could predict the value of a wine based on its taste without being told its value ahead of time. Though I am certainly no wine expert, I find it very challenging myself to distinguish whether a wine is "expensive" or "cheap".

But then again, I am just one person. There is evidence that suggests that there is some sort of wisdom that is inherent to a crowd. An entire book was actually written on this subject.

So, what would happen if you put a crowd of people together with six somewhat arbitrary wines and asked them to taste them and predict how much each cost? Would they be able to predict the value of the wine on average or would they be just as clueless as I am?

Motivated by seeking out the answer to these questions, I organized my wine tasting party - for social purposes and for science purposes!

• The Wine Tasting

My data set is made up of 10 participants. During the wine tasting, participants were given the option of filling out a survey that I had prepared for them. The survey consisted of an empty table that allowed participants to fill in with values.

The first column on the survey table indicated the number wine that each row in the table corresponded to. Wine numbers were listed along with actual wines that could be tasted, and participants were responsible for putting their predicted value for each wine with the correct wine number on their surveys.

There is a chance then that some data points could have been marked incorrectly, but I do not foresee this altering the data significantly.

The data table was also made up of a nominal value ($) column in which participants would place their predictions. To the right of that column was a Location column, which was used as a dummy variable (but also interesting). Finally, participants could also rank their wines and write notes in the right-most column. Figure 1 depicts the survey before it had been filled out.

FIGURE 1

Each participant was uniquely identified by an ID that was written on the back of the survey. IDs were randomly assigned before the wine tasting party commenced. The unique IDs are given in figure 2.

Each column in figure 2 labeled "Wine _" represents a different wine variety and value. It goes from "Wine 1" to "Wine 6" when scanning the table from left to right.

The mean, median, and mode of the individual's responses are calculated at the bottom of the table and are labeled as such in the left most column. Finally, at the very bottom of the table is the actual value of each wine. The listed values are in currency units which are not depicted in the table. The currency is 2010 US Dollars.

FIGURE 2

The most significant comparison that I wanted to make was between the mean (the average) predictions of the group against the actual value of the wine. Comparing the average predicted values with the actual values effectively tests my general question regarding how perception coincides with reality (at least with the value of wine tasting).

I have depicted the average predicted value versus the actual value in figure 3. The x-axis is segmented by corresponding wine number and the y-axis is the value of the wine in 2010 US Dollars.

FIGURE 3

• Discussion

Taking a look at the data, I am quite impressed with how the group performed. If we take a deeper look at figure 3 and qualitatively describe it in terms of the general trend, I see that as actual values fall so do predicted values, and as actual values rises, so do predicted values. This is strong evidence to suggest that people, on average, can distinguish the value of a wine simply via taste.

If we compare Wine 1 to Wine 2 for instance, we see that the actual value declines by 35% while predicted value similarly declines by 55%. While the degree is comparatively larger in the decline of the predicted value, the overall trend of rising or falling value is remarkably similar and is the case for five out of the six wines.

I think that even beyond predicting actual value of the wine, this trend following of the actual value of the wine is striking.

The only wine for which this was not the case was the final wine sampled, Wine 6. Wine 6 was in fact the least "valuable" wine but was ranked as the second most "valuable" wine. I think that this was the case because of the cognitive bias of anchoring. The most valuable wine, both actual and predicted, was sampled just prior to having the sixth wine and this anchor may have biased the participants overall when making their predictions of the value of wine 6.

Overall, it was a pretty rad wine tasting party and I think fun was had by all.

• Summary

Participants were asked to predict values of different wines. On average, the group of participants was able to fluctuate their wine prediction in tandem with the actual value of the wine in all but one case. The results indicate that a group of individuals are able to discern the nominal value of a wine simply based on taste.