Models under scrutiny as coronavirus gets more politicized

Models that estimate the rapid spread or quick extinction of the coronavirus have become the latest partisan flashpoint in a politicized pandemic that has Americans searching for answers — and finding sharply contrasting information.

The impressions those models have left are reminiscent of polling that showed former Secretary of State Hillary Clinton attracting more support than President Trump in the run-up to the 2016 presidential election. Clinton won the popular vote, but Trump won the White House, giving an easy and endless retort to those who are skeptical of political punditry.

Now, it is models of coronavirus case counts that are drawing skepticism and ire. Conservatives contend that they are exaggerating the threat posed by the pandemic, spurring an economic catastrophe that will be worse than the virus itself. Those on the left see an administration downplaying the health risk and cherry-picking models that are hopelessly optimistic.

“You’ll often see in public health or in politics declarations of victory if things aren’t as bad as the models said,” said Rich Besser, a former acting director of the Centers for Disease Control and Prevention (CDC) who now heads the Robert Wood Johnson Foundation. “Models are tools. Models are not reality.” 

It has not helped that the model most widely touted by the White House, produced by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington, has proven itself so deeply incorrect that it has drawn criticism even from fellow statisticians at the school.

That model has managed to be at once far too pessimistic, projecting that some states would need 10 or 20 times as much hospital capacity than they actually needed, and too optimistic, showing the number of cases and deaths plunging at an unrealistic rate.

Several states used the IHME model to build out field hospitals to prepare for a predicted surge in coronavirus patients, or to spend millions of dollars on respirators and ventilators they thought they would need. Many of the field hospitals have already been dismantled, and the ventilators sit unused, as the model proved too alarmist. Senior advisers to two Republican governors told The Hill they had been frustrated by what they now see as misleading information.

But models, not unlike political polls, are not meant to predict the future. Instead, both are snapshots in time, projections of what would happen under current circumstances or various scenarios.

A model’s output changes when the variables it considers change: If a model is told a country will practice no social distancing measures, it will forecast more coronavirus cases and deaths than if it is told a country will practice strict and enforced social distancing. A poll’s output changes when its variables are altered: If a pollster assumes that evangelical voters are more likely to turn out at higher rates, the Republican candidate will see their standing improve, and if younger voters or African Americans are disproportionately likely to show up to vote the Democratic candidate might benefit.

As a virus begins to spread, models can have huge ranges — their equivalent of a poll’s margin of error — because so much about a disease remains unknown.

Besser, who headed the CDC as the H1N1 influenza began to spread across the world more than a decade ago, recalled a model showing that on a 5-point scale the virus would be a 3, with a range of 1 to 5 — a range that included both no threat whatsoever and global catastrophic pandemic that would kill tens of millions. Once much more was known about the H1N1 virus, it became clear it was not the pandemic catastrophe it might have been.

“Early in a pandemic or any public health emergency, real data is lacking, and modeling can provide you with a framework in which to be thinking about critical factors that can influence the course of an outbreak,” Besser said. “But models are never a replacement for real data. As an outbreak progresses, models get better informed by reality.” 

On Monday, an internal estimate prepared by a researcher at Johns Hopkins University for the Federal Emergency Management Agency made headlines when it showed the number of potential new coronavirus cases could jump to 200,000 a day in just a few weeks, with thousands of Americans dying every day — an estimate so shocking that the White House disavowed it.

It was not the first time that a model caused a political headache. When the Ebola virus broke out in West Africa in 2014, an internal CDC model projected as many as 1.4 million people would be infected in just four months. That figure was the absolute worst-case scenario, assuming nothing at all would be done to stop the virus’s spread — but it was a headline that television and newspaper reporters could not resist.

The number was so startling that senior public health officials including Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases, and Rajiv Shah, director of the U.S. Agency for International Development, both called then-CDC Director Tom Frieden to complain. Frieden refused to relent, and newspapers ran their headlines the next day.

“I always joke that [models are] not to scare people out of their wits, it’s to scare them into their wits,” said Michael Osterholm, director of the Center for Infectious Disease Research and Prevention at the University of Minnesota.

Epidemiologists and policymakers say models are valuable in understanding what might happen, given different circumstances, and how to allocate resources. In the case of a virus, models would predict where it would begin to spread, and policymakers and public health officials could rush equipment or personnel to impacted areas in time.

“They’re really helpful for projections so that health care systems don’t get overwhelmed. The real disasters internationally have been when there’s so much COVID-19 that hospitals can’t respond and you have ICU rationing and all kinds of other things that are terrible,” said Paul Sax, clinical director of the division of infectious diseases at Brigham and Women’s Hospital in Boston. 

No political strategist relies solely on one poll or pollster. Similarly, no one at the CDC, the World Health Organization or any other public health agency relies on only one model. CDC officials consider dozens of models, some of which are developed in-house and others which come from universities and experts around the world.

And every model carries its own flaws. Models cannot account for some factors that influence a pandemic spread, Besser said, like racial data that shows African Americans being hit disproportionately hard by the coronavirus in part because of a lack of access to quality health care.

Perhaps the most important role models can play is to convey to both policymakers and the general public just how dangerous a virus might become if drastic actions are not taken. Large and frightening numbers of potential casualties are one way public health officials can grab attention when they need to. 

“You will always be accused of either not doing enough or doing too much. And early on, you definitely want to fall into the camp of having done too much,” Besser said. “Because you save lives that way.”