COVID-19 predictive models aren't destined to come true

As more states prepare to lift stay at home orders, the conversation has again focused on predictive models. A recent model in a draft government report forecasts that covid-19 cases will surge by June 1, with the potential for 3,000 deaths a day.

But models keep changing their predictions, sometimes abruptly. Within the span of a couple of weeks in April, a leading covid-19 model maintained by the Institute for Health Metrics and Evaluation (IMHE) reduced the expected number of casualties from 100,00-240,000 to 70,000 only to move the needle upward again this week, to 135,000. Another influential model, designed by The Imperial College in London, proposed scenarios for the United Kingdom that varied between 500,000 victims, for a worst-case scenario, to no more than 20,000, under lock-down conditions.

What accounts for such discrepancies? Do the differences show theoretical or mathematical errors?

In both cases the problem was not in the predictions themselves, but in our ability to forecast both the evolution of the epidemic and our response to it. In times of crisis, people want hard numbers to foretell the future. But theoretical models provide probabilities, not certainties. Thus, the most effective way to reduce uncertainty in pandemic emergencies is to focus primarily on practical action. History teaches us as much.

For example, in 1665 London also thirsted for certainty. After just returning to predictable life following long years of civil strife, the city received the visit of another old enemy: the plague. The unwanted guest had made appearances in Amsterdam and Rotterdam between 1663 and 1664, but Londoners, many of whom were both fervently religious and reasonably open to empirical inquiry, hoped that their island kingdom was sheltered by Nature and God alike. Moreover, they thought that the knowledge of cold facts would help them avoid the disaster.

The first two victims, two stray Frenchmen, died in Drury Lane in late November, 1664. Yet, soon, born and bred Londoners also started falling ill. Daniel Defoe, the famous author of "Robinson Crusoe," recalled in "A Journal of the Plague Year" how the educated classes of the city feverishly followed news about burials. The numbers, they hoped, would free their minds from error and superstition. Good or bad, the figures would surely tell them where they stood soon enough. If they went up, they would run away. If numbers stayed put, so would they.

Yet, the numbers made many reasonable minds more confused than certain.

For example, the tally of the burials recorded with actuarial precision by Defoe for one of the neighborhoods first afflicted, St. Giles-in-the-Fields (not far from Leicester Square), seemed to have a mind of their own. First, they crept up. Sixteen burials in the week after Christmas (December 27 to January 3) were followed by a hope-giving 12 deaths by January 10, only to be dashed by the steady march upward in the three weeks to come: 18 victims by January 17, 23 by January 24 and 24 by January 27. Yet, the deaths went down to 21 the next week and stayed at 24 until February 14, 1665. It seemed that death was playing hide and seek in London.

Defoe also recalled that while the overall number of deaths increased between March-May 1665, in some parishes they actually went down. Yet, even when the figures did go up, the local aldermen who put their wishes ahead their fears hastened to blame the demises on the deceivingly called "spotted fever," probably the same devil by another name.

By June 1665, as uncertainty turned into full-blown anxiety, the thirst for assurance intensified, and an outburst of publications met the demand for more information. Despite the Licensing Act of 1662 that tried to rein in "the frequent Abuses in printing seditious treasonable and unlicensed Books and Pamphlets," printers whipped by anxiety and censors who preferred to turn a blind eye in a time of crisis created a pre-modern version of social media and misinformation.

Pamphlets, at times no longer than a Medium essay, scared the world witless. "Gadbury's Astrological Predictions," "Poor Robin's Almanack," "Come out of her, my People, lest you be Partaker of her Plagues," "Fair Warning" and "Britain's Remembrancer" foretold the ruin of the city by a certain appointed time and in a certain manner.

The walls and fences of London were also covered with posters and the ground was littered with discarded handbills. Each carried a recognizable logo: Merlin's cap. The services offered were astrological predictions. Astutely, Defoe observed that the quest for astrological wisdom was fueled not by superstition alone, but also by the reasoned belief that there should be a certain calculable method by which one could inquire about vitally important personal matters.

In the end, neither the cold calculation of the deaths, nor the astrological models did much to stop the disease. The numbers initially gave people a false sense of control only to fail them later. The astrological models could only disappoint when prediction upon prediction failed, either positively or negatively, to match reality.

What did work? Practical wisdom.

Particularly effective were the ambulatory teams of examiners that used a sort of "test and trace" routine to identify new cases, quarantine afflicted houses and provide support in the form of watchmen and nurses to care for those they locked up. Domesticated animals were curbed and possessions were impounded. Travelers were quarantined and the border with Scotland closed. Precaution and simple public health campaigns worked.

After peaking in August 1665 at 31,159 dead, the epidemic dwindled off by the first frosts in November. During the Great Plague a total of 68,596 Londoners died, or about 15 percent of the city population.

Then, as now, people suffering through a dramatic epidemic sought objective statistics and made reasonable (yet, invalid) attempts to predict the span and impact of the affliction. However, it was not the theoretical control of the problem in its enormity that brought the pestilence to heel, but the no-nonsense method of managing that which could be managed: isolating and supporting the sick, controlling the panic, eliminating the causes of the infection and mobilizing the examiners, watchmen and burying parties that kept the epidemic at bay. All this was done by the local authorities, especially the parish councils. In Defoe's account, the King was a remote presence, having taken refuge at Oxford. The Londoners, the Lord Mayor and his council were the main actors of the drama.

In our own hour of pandemic emergency, we also seek certainty in theoretical modeling of covid-19's spread. Although our tools are more refined that those of the 17th century, their variability is still great — as the University of Washington and Imperial College examples demonstrate. Diffusion curves seem to be better at predicting what already happened than what will happen. Now, as more than 350 years ago, we should use theoretical models not to fuel our worst fears but to check the outcome of our best practical efforts.

Matei is professor of communication at Purdue University. This piece was written for The Washington Post.