Prediction is a difficult business. It’s not just a matter of anticipating the timing of a terrorist attack; governments also have to consider the method that may be used, and the potential impact that method will have. In the weeks following the 9/11 attacks in 2001, several people in the US media and Congress received letters containing toxic anthrax bacteria. It led to five deaths, raising concerns that other bioterrorist attacks may follow.[59] One of the top threats was thought to be smallpox. Despite having been eradicated in the wild, samples of the virus were still stored in two government labs, one in the US and one in Russia. What if other, unreported, smallpox viruses were out there and fell into the wrong hands?
Using mathematical models, several research groups tried to estimate what might happen if terrorists released the virus into a human population. Most concluded that an outbreak would grow quickly unless pre-emptive control measures were in place. Soon after, the US Government decided to offer half a million healthcare workers vaccination against the virus. There was limited enthusiasm for the plan: by the end of 2003, fewer than 40,000 workers had opted for the vaccine.
In 2006, Ben Cooper, then a mathematical modeller at the UK Health Protection Agency, wrote a high-profile paper critiquing the approaches used to assess the smallpox risk. He titled it ‘Poxy Models and Rash Decisions’. According to Cooper, several models included questionable assumptions, with one particularly prominent example. ‘Collective eyebrows were raised when the Centers for Disease Control’s model completely neglected contact tracing and forecast 77 trillion cases if the epidemic went unchecked,’ he noted. Yes, you read that correctly. Despite there being fewer than 7 billion people in the world at the time, the model had assumed that there were an infinite number of susceptible people that could become infected, which meant transmission would continue indefinitely. Although the CDC researchers acknowledged it was a major simplification, it was bizarre to see an outbreak study make an assumption that was so dramatically detached from reality.[60]
Still, one of the advantages of a simple model is that it’s usually easy to spot when – and why – it’s wrong. It’s also easier to debate the usefulness of that model. Even if someone has limited experience with mathematics, they can see how the assumptions influence the results. You don’t need to know any calculus to notice that if researchers assume a high level of smallpox transmission and an unlimited number of susceptible people, it can lead to an unrealistically large epidemic.
As models become more complicated, with lots of different features and assumptions, it gets harder to identify their flaws. This creates a problem, because even the most sophisticated mathematical models are a simplification of a messy, complex reality. It’s analogous to building a child’s model train set. No matter how many features are added – miniature signals, numbers on the carriages, timetables full of delays – it is still just a model. We can use it to understand aspects of the real thing, but there will always be some ways in which the model will differ from the true situation. What’s more, additional features may not make a model better at representing what we need it to. When it comes to building models, there is always a risk of confusing detail with accuracy. Suppose that in our train set all the trains are driven by intricately carved and painted zoo animals. It might be a very detailed model, but it’s not a realistic one.[61]
In his critique, Cooper noted that other, more detailed smallpox models had come to similarly pessimistic conclusions about the potential for a large outbreak. Despite the additional detail, though, the models still contained an unrealistic feature: they had assumed that most transmission occurred before people developed the distinctive smallpox rash. Real life data suggested otherwise, with the majority of transmission happening after the rash appeared. This would make it much easier to spot who was infectious, and hence control the disease through quarantine rather than requiring widespread vaccination.
From disease epidemics to terrorism and crime, forecasts can help agencies plan and allocate resources. They can also help draw attention to a problem, persuading people that there is a need to allocate resources in the first place. A prominent example of such analysis was published in September 2014. In the midst of the Ebola epidemic that was sweeping across several parts of West Africa, the CDC announced that there could be 1.4 million cases by the following January if nothing changed.[62] Viewed in terms of Nightingale-style advocacy, the message was highly effective: the analysis caught the world’s attention, attracting widespread media coverage. Like several other studies around that time, it suggested that a rapid response was needed to control the epidemic in West Africa. But the CDC estimate soon attracted criticism from the wider disease research community.
One issue was the analysis itself. The CDC group behind the number was the same one that had come up with those smallpox estimates. They’d used a similar model, with an unlimited number of susceptible people. If their Ebola model had run until April 2015, rather than January, it would have estimated over 30 million future cases, far more than the combined populations of the countries affected.[63] Many researchers questioned the appropriateness of using a very simple model to estimate how Ebola might be spreading five months later. I was one of them. ‘Models can provide useful information about how Ebola might spread in the next month or so,’ I told journalists
