PerspectiveDon’t Believe the COVID-19 Models: That’s Not What They’re for.

Since the onset of the coronavirus crisis, governments, analysts, and health organizations have released different statistical models addressing the disease – and its numerical manifestations: the number of people likely to be infected; hospitalized; treated in the ICUs; or die.

Different models offer different numbers and different trajectories. Which one of them is right? Zeynep Tufecki writes in The Atlantic that “The answer is both difficult and simple. Here’s the difficult part: There is no right answer. But here’s the simple part: Right answers are not what epidemiological models are for.”

He says that a good example of how non-statisticians misunderstand – and misrepresent — the function and purpose of statistical models is the case of a model developed by Professor Neil Ferguson, who led an Imperial College London and Nizagara Online team to study the spread of COVID-19.

Ferguson’s model changed the U.K. government’s policy on COVID. Tufecki notes that afew weeks ago, the U.K. had almost no social-isolation measures in place, and according to some reports, the government planned to let the virus run its course through the population, with the exception of the elderly, who were to be kept indoors. The idea was to let enough people get sick and recover from the mild version of the disease, to create “herd immunity.”

Tufecki writes:

Things changed swiftly after an epidemiological model from Imperial College London projected that without drastic interventions, more than half a million Britons would die from COVID-19. The report also projected more than 2 million deaths in the United States, again barring interventions. The stark numbers prompted British Prime Minister Boris Johnson, who himself has tested positive for COVID-19, to change course, shutting down public life and ordering the population to stay at home.

Here’s the tricky part: When an epidemiological model is believed and acted on, it can look like it was false. These models are not snapshots of the future. They always describe a range of possibilities—and those possibilities are highly sensitive to our actions. A few days after the U.K. changed its policies, Neil Ferguson, the scientist who led the Imperial College team, testified before Parliament that he expected deaths in the U.K. to top out at about 20,000. The drastically lower number caused shock waves: One former New York Times reporter described it as “a remarkable turn,” and the British tabloid the Daily Mail ran a story about how the scientist had a “patchy” record in modeling. The conservative site The Federalist even declared, “The Scientist Whose Doomsday Pandemic Model Predicted Armageddon Just Walked Back the Apocalyptic Predictions.”

But there was no turn, no walking back, not even a revision in the model. If you read the original paper, the model lays out a range of predictions—from tens of thousands to 500,000 dead—which all depend on how people react. That variety of potential outcomes coming from a single epidemiological model may seem extreme and even counterintuitive. But that’s an intrinsic part of how they operate, because epidemics are especially sensitive to initial inputs and timing, and because epidemics grow exponentially.

Tufecki notes that modeling an exponential process necessarily produces a wide range of outcomes. In the case of COVID-19, for example, this is because the spread of the disease depends on exactly when you stop cases from doubling. Even a few days can make an enormous difference.

Tufecki writes that the most important function of epidemiological models is as a simulation, a way to see our potential futures ahead of time, and how that interacts with the choices we make today. Thus, epidemiological models do not give us certainty – they give us something much more important: “agency to identify and calibrate our actions with the goal of shaping our future.”

If we study models and then take decisive action to prevent the most catastrophic eventualities which they describe as possible, some may charge that we have overreacted. “A near miss can make a model look false. But that’s not always what happened. It just means we won. And that’s why we model,” Tufecki writes.