MINI CASE 12.1
The importance of intuition is generally accepted. Intuition can give insight and a sense of new direction but can mislead and be over-optimistic. Imagine you are lost in a fog. Intuition may tell you to continue, turn back or give some other option. You may not know how good these are until after the event. A compass will always give the direction of magnetic north. How useful the compass is will depend on your knowledge of the area and your experience of using a compass. The old saying is that the compass will never lie. There is a judgement in using what you may expect and what you calculate. Both can give good results or be misleading. What we do need to be aware of are the ways we can be wrong. In the fog, we can wrongly think we know the best direction. In the fog, others might agree with a poor decision. Having made a decision, we can also be over-confident that it is right. Like a compass, confidence intervals should give a sense of direction with the data we are working with.
Financial Times, 29 August 2003
Many marketing people make this mistake say Andrew Gersh off and Eric Johnson. But did FT readers fare any better:
Marketing is a social process, not simply the analysis of dry statistical reports. Whether your customers live in another country or another region, or are part of a different economic class, the heart of marketing is under-standing them and their needs. Often, though, marketers and managers are very different from their customers - most products are developed by teams of like-minded people who may have little in common with the intended buyer.
This can create problems for marketers during the decision-making process - problems that have been highlighted by consumer research, behavioural economics and social psychology. Here we examine two of these pitfalls and discuss their causes and possible cures. Moreover, we shall see how a sample of Financial Times readers performed in making judgements like the ones we discuss.
False consensus
In a 1993 study, US managers were asked to estimate various attributes of markets, including the percentage of beer sold in US supermarkets that was imported from other countries and the percentage of US households that purchased canned chilli. They were also asked how much they personally liked and purchased imported beer and canned chilli.
At the time of the study only about 2 per cent of beer sold in US supermarkets was imported. The executives, who tended to like and buy imported beer, gave an average estimate of 20 per cent. On average, the more an executive liked and purchased imported beer, the higher was his or her estimate of the amount of imported beer sold.
Canned chilli, on the other hand, is a product that was largely disliked and was rarely purchased by US executives. While 40 per cent of US households buy it in a given year, the executives' average estimate was only 28 per cent. Again, the more an executive personally shied away from canned chilli, the lower was his or her estimated purchase for the country as a whole.
The explanation for this disparity lies in a psychological phenomenon called the false consensus effect, in which people tend to think that their own attitudes are more common than they really are. When people estimate what others like and do, their own attitudes sway their responses. Since the psychologists Lee Ross, David Green and Pamela House identified this effect in 1977, a number of reasons have been suggested as to why it occurs.
One is that it is so easy for people to think of what they like and dislike that they give these preferences extra weight. Another is that when they think of other people, they think of people they know well, who tend to be similar to them. Thus their judgement is biased.
What can managers do to avoid false consensus in their estimates? Just being aware of it may not be enough. Joachim Kruger and Russell Clement, psychological researchers, found the effect can occur even when people are specifically warned about it.
But there are ways in which the effect may be reduced. Perhaps the most important is by using market research. A number of studies have found that observing real data can reduce the incidence of false consensus. Second, managers can employ diverse teams of people.
Over-confidence
Over-confidence The second error lies in failing to identify what we do not know. When we asked the executives to indicate how certain they were of their estimates, they were typically over-confident.
Along with estimates, they also provided 90 per cent confidence intervals (or upper and lower bounds for their estimates), representing their belief that the true value would fall, on average, nine out of ten times within these bounds.
Over-confidence is not universal but it does appear to be common. The best way to guard against it is to make explicit statements about how confident you are and to check how things turn out. You may start out being just as over-confident as our executives but at least you have a chance to learn from your mistakes.
Sophisticated FT readers
So how do readers of the Financial Times compare with other executives? Why would we expect them to be different? In 1997 Richard Thaler asked FT readers to solve a popular problem from game theory, sometimes known as the Beauty Contest game.
Professor Thaler wanted to see how they would do in comparison to the other groups that had played the game.
The bottom line was this: they showed a high level of strategic sophistication. Whether it was due to a significant prize, self-selection (only those who thought they would do well would enter) or other characteristics of FT readers, this group was more sophisticated than many others, including samples of chief executives, financial analysts and MBAs at many of the world's top institutions.
Would the same thinking apply here? As a group, the answer is no.
We asked 274 respondents questions similar to those we had used in the past, but which we thought might tap differences between European (particularly those from the UK) and US readers. We also offered a $250 prize funded jointly by the Columbia Centre for the Decision Sciences and the Columbia Centre for Excellence in E-Business.
First, we asked readers to estimate the percentage of European beer sold in US supermarkets. According to Information Resources, which supplied recent data, European imports account for 4.1 per cent of the US market; our FT readers gave an average estimate of 24.4 per cent, more than five times as high.
Our second question asked about the average annual sum spent on tea (in bag form) Research gives a figure of $5.92 but our respondents estimated on average more than $56, off by a factor of almost ten. These are difficult questions and FT readers may have known the perils of intuitive estimation. If so, they should have spread their confidence intervals wide, yielding - as we had hoped - answers outside their confidence intervals (that is to say, surprises) only 10 per cent of the time. However, 82 per cent of the confidence intervals for beer and 72 per cent of those for tea did not contain the right answer.
Why did this occur? One hint is that European beers made up 45 per cent of the respondents' purchases and they reported spending about $50 on tea bags every year. Once more, it looks as if FT readers thought their customers were much more like them than they really are - in spite of the incentive offered by the prize.
Of course, this task might have seemed more difficult for Europeans; and they might have felt at a handicap compared with our US entrants. They need not have worried: while they drank more European beer and purchased more tea-bags than their US counterparts, it did not affect their judgements significantly. They did no worse or better than their US counterparts.