“Sooner or later we are going to have to have an inquiry into the role of statisticians in the Covid-19 crisis,” declared the journalist Ross Clark in the Daily Telegraph recently. “They will have to be put in the hot seat and grilled as to whether they contributed to the sense of panic which has gripped so many people and disrupted normal life for so long.” As one of those beastly statisticians, I would relish the opportunity.
In my book The Art of Statistics, published last year, I quote the statistician and writer Nate Silver, who said: “Numbers have no way of speaking for themselves… we imbue them with meaning.” During this crisis, a lot of meaning has been imbued by people who want the numbers to support their argument. This may include making comparisons with Sweden to say a strict lockdown is unnecessary, or pointing to rising infection rates to demand a zero-Covid strategy. In contrast, statisticians tend to be pedantically cautious in their interpretation of data, as they understand its frailties and uncertainties, and are very reluctant to use it to argue that something should or should not be done.
Statisticians also shouldn’t be blamed for the government’s unmourned daily press briefings during the first months of the pandemic lockdown. These revealed how politicians love to throw around lots of big numbers, showing pretty graphs, all led by the No 10 communications team. Back in May I called this “number theatre”. The frailty of some of these numbers is now being revealed: the accumulating Covid-19 death count in England dropped in mid-August by about 5,000, as the overall figure now doesn’t include anyone who died more than 28 days after a positive test. This decision was not apparently made by statisticians, and maybe more for unity of measurement between the four nations of the United Kingdom, than for its appropriateness.
Numbers may not measure what you think they may be measuring, but they are rarely completely wrong. That’s why the statistical community let out a collective gasp last Sunday when Stephen Hahn, the commissioner of the US Food and Drug Administration, told a press conference that treatment with convalescent plasma would mean that of “100 people who were sick with Covid-19, 35 would have been saved”. If true, this would be one of the most remarkable medical discoveries in history, saving one life for every three people treated. But it is completely false. In fact 8.9% of the treated group died, compared with 13.7% of a comparison group, a 35% relative difference. The absolute difference was 5%, so if we believed the figures, in fact 20 people had to be treated to save one life. But we should not even believe those numbers, being based on selected patients receiving the treatment, and not comparing like with like in a proper randomised design.
This is possibly the most statistically dim-witted utterance I have witnessed from any authority during this crisis, and he’s up against some pretty stiff competition, not least from the president flanking him at the press conference.
People tend to have an exaggerated sense of their vulnerability to Covid-19, and I admit that statisticians must take some responsibility for not communicating a realistic idea of the risks. Partly this is a problem with averages – just saying that around 1% of people die after catching the virus can be misleading when this average figure disguises a huge variability, estimated to range from around 20% in the very old and frail, to roughly one in 50,000 for schoolchildren. It is quite right to focus on the effects of ethnicity, deprivation and people-facing jobs, each of which can roughly double the risk of catching and dying from the virus. But being just six years older also doubles your risk: it goes up, like some lethal form of compound interest, at around 12% for each year of age.
The spread of the virus at the start of the epidemic showed how difficult it was to grasp this idea of exponential increase, but it may help us understand why the current rise in cases is not being followed by extra hospital admissions or deaths. It is vital that we don’t just count cases, but look at their ages, and the latest report from Public Health England reveals the highest rate of positive tests is in the 15-44 age-group. If 5,000 30-year-olds get infected, we might expect just one or two to die, and so this helps explain the lack of severe consequences so far. But it’s a different picture for the frail and elderly, who need proper protection.
This is a tricky time for us all, with a need to balance potential harms and benefits while faced with so much uncertainty. This is true both for society as a whole and for each individual weighing up their options. As a community, we require increasingly intensive coronavirus surveillance, and rapid and proportionate local action, combined with maximum efforts to open schools and get the economy moving, while also protecting the extremely vulnerable. And as individuals, we all have to face the fact that this is not going to go away in a hurry: even if a vaccine eventually arrives, it may not afford complete protection and may need repeating. So those who are currently shielding will at some point have to decide whether they want to be shut in their homes for the foreseeable future. This process will be gradual, and suitably cautious, but we have long since learned to manage risks such as transport accidents and terrorism.
This transition to a world that is not “safe”, but hopefully low risk, will be driven by statistics. And overall, I think the statisticians have done a good job through the crisis. But I would say that, wouldn’t I? I must admit to a conflict of interest, being a non-executive director of the UK Statistics Authority, which oversees the national statistics and, fortunately, answers to parliament rather than a ministry. So bring on that hot seat and the grilling.
• David Spiegelhalter is chair of the Winton Centre for Risk and Evidence Communication at Cambridge University. He is the author of The Art of Statistics