PandemicsWho Was Better at Predicting the Course of the Pandemic – Experts or the Public?
Despite the potential influence of both expert and non-expert predictions on people’s responses to the pandemic, there’s been limited research on the accuracy of either – or indeed on the difference in accuracy between them. Perhaps it’s not surprising that most people’s best guesses about the number of deaths and infections were off: predictions about emerging diseases are hard, and none of us has a crystal ball. We found that even experts weren’t particularly good at predicting the pandemic’s ultimate course and impact. But our level of confidence about our predictions is within our control – and the evidence suggests that most of us could stand to be a bit more humble.
Early on in the pandemic, it seemed as if the media was asking anyone with potentially relevant expertise – scientists, doctors, statisticians – to tell us what was coming. These individuals were frequently asked to give off-the-cuff answers to questions about how bad the pandemic might get, even though there was little data to go on.
The use of expert predictions like these are important. They have the potential to shape public opinion and policy and influence how events unfold. Yet these have often been disregarded, particularly on social media, where alternative predictions by non-experts (including misinformation about experts’ forecasts) spread easily.
Despite the potential influence of both expert and non-expert predictions on people’s responses to the pandemic, there’s been limited research on the accuracy of either – or indeed on the difference in accuracy between them. To this end, in April 2020 my colleagues and I conducted an experiment to find out whether experts really did have a better idea of what was on the way than the rest of us.
Having a sense of this could inform what sort of public role we want experts to play in a future pandemic. Likewise, it could suggest how much weight we should place on expert and non-expert predictions of how future disease outbreaks will unfold.
Higher Accuracy, Lower Confidence
We asked 140 experts (epidemiologists, statisticians, mathematical modelers, virologists and clinicians) and 2,086 laypersons to give their best guesses on several questions about how the pandemic would progress.
We asked them, by the end of 2020, how many people in the U.K. would have been infected with COVID-19, how many deaths there would have been in the U.K., and how many people would have died out of every 1,000 infected with the virus in the U.K. and worldwide. Here’s how the two groups fared.
The experts’ best guesses were more accurate than laypeople’s on every question, but even the experts underestimated the total number of infections and deaths by a substantial margin. For example, the median estimate for the number of U.K. COVID-19 infections by the end of 2020 was 250,000 for non-experts and 4 million for experts. Calculations based on infection-fatality ratio research suggest the true count was closer to 6.4 million.
