The Black Swan: The Impact of the Highly Improbable

Much of the empirical research agrees with this pattern of overestimation and underestimation of Black Swans. Kahneman and Tversky initially showed that people overreact to low-probability outcomes when you discuss the event with them, when you make them aware of it. If you ask someone, “What is the probability of death from a plane crash?” for instance, they will raise it. However, Slovic and his colleagues found, in insurance patterns, neglect of these highly improbable events in people’s insurance purchases. They call it the “preference for insuring against probable small losses” – at the expense of the less probable but larger impact ones.

Finally, after years of searching for empirical tests of our scorn of the abstract, I found researchers in Israel that ran the experiments I had been waiting for. Greg Barron and Ido Erev provide experimental evidence that agents underweigh small probabilities when they engage in sequential experiments in which they derive the probabilities themselves, when they are not supplied with the odds. If you draw from an urn with a very small number of red balls and a high number of black ones, and if you do not have a clue about the relative proportions, you are likely to underestimate the number of red balls. It is only when you are supplied with their frequency – say, by telling you that 3 percent of the balls are red – that you overestimate it in your betting decision.

I’ve spent a lot of time wondering how we can be so myopic and short-termist yet survive in an environment that is not entirely from Mediocristan. One day, looking at the gray beard that makes me look ten years older than I am and thinking about the pleasure I derive from exhibiting it, I realized the following. Respect for elders in many societies might be a kind of compensation for our short-term memory. The word senate comes from senatus, “aged” in Latin; sheikh in Arabic means both a member of the ruling elite and “elder”. Elders are repositories of complicated inductive learning that includes information about rare events. Elders can scare us with stories – which is why we become overexcited when we think of a specific Black Swan. I was excited to find out that this also holds true in the animal kingdom: a paper in Science showed that elephant matriarchs play the role of superadvisers on rare events.

We learn from repetition – at the expense of events that have not happened before. Events that are nonrepeatable are ignored before their occurrence, and overestimated after (for a while). After a Black Swan, such as September 11, 2001, people expect it to recur when in fact the odds of that happening have arguably been lowered. We like to think about specific and known Black Swans when in fact the very nature of randomness lies in its abstraction. As I said in the Prologue, it is the wrong definition of a god.

The economist Hyman Minsky sees the cycles of risk taking in the economy as following a pattern: stability and absence of crises encourage risk taking, complacency, and lowered awareness of the possibility of problems. Then a crisis occurs, resulting in people being shell-shocked and scared of investing their resources. Strangely, both Minsky and his school, dubbed Post-Keynesian, and his opponents, the libertarian “Austrian” economists, have the same analysis, except that the first group recommends governmental intervention to smooth out the cycle, while the second believes that civil servants should not be trusted to deal with such matters. While both schools of thought seem to fight each other, they both emphasize fundamental uncertainty and stand outside the mainstream economic departments (though they have large followings among businessmen and nonacademics). No doubt this emphasis on fundamental uncertainty bothers the Platonifiers.

All the tests of probability I discussed in this section are important; they show how we are fooled by the rarity of Black Swans but not by the role they play in the aggregate, their impact. In a preliminary study, the psychologist Dan Goldstein and I subjected students at the London Business School to examples from two domains, Mediocristan and Extremistan. We selected height, weight, and Internet hits per website. The subjects were good at guessing the role of rare events in Mediocristan-style environments. But their intuitions failed when it came to variables outside Mediocristan, showing that we are effectively not skilled at intuitively gauging the impact of the improbable, such as the contribution of a blockbuster to total book sales. In one experiment they underestimated by thirty-three times the effect of a rare event.

Next, let us see how this lack of understanding of abstract matters affects us.

The Pull of the Sensational

Indeed, abstract statistical information does not sway us as much as the anecdote – no matter how sophisticated the person. I will give a few instances.

The Italian Toddler. In the late 1970s, a toddler fell into a well in Italy. The rescue team could not pull him out of the hole and the child stayed at the bottom of the well, helplessly crying. Understandably, the whole of Italy was concerned with his fate; the entire country hung on the frequent news updates. The child’s cries produced acute pains of guilt in the powerless rescuers and reporters. His picture was prominently displayed on magazines and newspapers, and you could hardly walk in the center of Milan without being reminded of his plight.

Meanwhile, the civil war was raging in Lebanon, with an occasional hiatus in the conflict. While in the midst of their mess, the Lebanese were also absorbed in the fate of that child. The Italian child. Five miles away, people were dying from the war, citizens were threatened with car bombs, but the fate of the Italian child ranked high among the interests of the population in the Christian quarter of Beirut. “Look how cute that poor thing is”, I was told. And the entire town expressed relief upon his eventual rescue.

As Stalin, who knew something about the business of mortality, supposedly said, “One death is a tragedy; a million is a statistic”. Statistics stay silent in us.

Terrorism kills, but the biggest killer remains the environment, responsible for close to 13 million deaths annually. But terrorism causes outrage, which makes us overestimate the likelihood of a potential terrorist attack – and react more violently to one when it happens. We feel the sting of man-made damage far more than that caused by nature.

Central Park. You are on a plane on your way to spend a long (bibulous) weekend in New York City. You are sitting next to an insurance salesman who, being a salesman, cannot stop talking. For him, not talking is the effortful activity. He tells you that his cousin (with whom he will celebrate the holidays) worked in a law office with someone whose brother-in-law’s business partner’s twin brother was mugged and killed in Central Park. Indeed, Central Park in glorious New York City. That was in 1989, if he remembers it well (the year is now 2007). The poor victim was only thirty-eight and had a wife and three children, one of whom had a birth defect and needed special care at Cornell Medical Center. Three children, one of whom needed special care, lost their father because of his foolish visit to Central Park.

Well, you are likely to avoid Central Park during your stay. You know you can get crime statistics from the Web or from any brochure, rather than anecdotal information from a verbally incontinent salesman. But you can’t help it. For a while, the name Central Park will conjure up the image of that that poor, undeserving man lying on the polluted grass. It will take a lot of statistical information to override your hesitation.

Motorcycle Riding. Likewise, the death of a relative in a motorcycle accident is far more likely to influence your attitude toward motorcycles than volumes of statistical analyses. You can effortlessly look up accident statistics on the Web, but they do not easily come to mind. Note that I ride my red Vespa around town, since no one in my immediate environment has recently suffered an accident – although I am aware of this problem in logic, I am incapable of acting on it.

Now, I do not disagree with those recommending the use of a narrative to get attention. Indeed, our consciousness may be linked to our ability to concoct some form of story about ourselves. It is just that narrative can be lethal when used in the wrong places.

THE SHORTCUTS

Next I will go beyond narrative to discuss the more general attributes of thinking and reasoning behind our crippling shallowness. These defects in reasoning have been cataloged and investigated by a powerful research tradition represented by a school called the Society of Judgment and Decision Making (the only academic and