There’s more than meets the eye in 2020 election forecasts

Election season is officially upon us. FiveThirtyEight has officially released Nate Silver’s 2020 forecast, and the Economist debuted its first-ever statistical forecast of a U.S. presidential race. 

Most discussions surrounding these forecasts focus on how data scientists reached their estimates. Forecasting a presidential election is never simple. And with many new economic and social uncertainties due to a global pandemic, 2020 presents more obstacles than ever. 

One could easily overlook changes in how forecasts are being communicated in 2020. But both FiveThirtyEight’s and the Economist’s forecast displays make one thing clear: Journalists are being extra careful about how they convey uncertainty in their predictions. This observation alone may tell us as much about the forecasts as the graphics. 

In 2016, the typical forecast display led with win probabilities, or percentages, such as Trump having a 28 percent chance. Now FiveThirtyEight and the Economist instead lead with frequencies. The Economist’s top line, for example, states that Biden has a 5 in 6 chance of winning versus Trump’s 1 in 6. Research suggests that using frequencies helps readers grasp probability. When faced with probability, it can be easier to round up (or down) than to form a clear conception of what a 30 percent chance means. We won’t have another Biden versus Trump election to test a model’s predictions, but we can imagine getting heads on a coin flip three out of ten times. 

For FiveThirtyEight, frequencies aren’t entirely new; they first used them in 2018. But 2020’s treatment goes further, introducing new frequency visualizations. For example, the page header is a grid of U.S. maps the reader must scroll past before seeing numeric estimates. Six of 22 total maps are red, showing combinations that would lead to Trump’s reelection. The remainder show combinations of blue states that would result in a Biden win. Making readers first experience the forecast as a visual task, FiveThirtyEight helps them internalize that a Biden win isn’t a foregone conclusion. 

These may not seem like big changes. But after years of emphasizing numbers, high-profile, self-proclaimed “nerds” like Silver recognize the importance of forecast communication alongside the quality of the model. That’s worth noting. 

The added emphasis on uncertainty this year also begs the question, why? Better communication may help the forecasters avoid blame. When a forecast is probabilistic, there’s always a chance that the less expected outcome will occur. Research suggests that emphasizing uncertainty makes readers less likely to blame the forecaster for getting it wrong. 

In fact, after widespread public disapproval of the 2016 forecasts, some outlets argued that the general public lacks competence to interpret probabilistic forecasts. By thinking carefully about readers’ needs, FiveThirtyEight and the Economist acknowledge these challenges. In particular, FiveThirtyEight’s forecast predicts a significantly tighter race than the Economist’s. This seems more strategic than afterthought. This year’s forecast page has a tighter narrative and simpler layout, with many model details relegated to separate pages.  

While making uncertainty clearer is important to good science communication, these new design choices are tricky to evaluate. Consider FiveThirtyEight’s introduction of Fivey Fox, who is perhaps the strongest indicator that Silver intends to emphasize uncertainty this year.  

A sign-holding cartoon fox in the page’s margins, Fivey Fox helps FiveThirtyEight communicate advice directly to readers. The most notable of Fivey’s messages so far have reminded readers of the potential for extreme outcomes: low probability events that run counter to the forecast’s overall trend. Any forecasting model involves assumptions, and when these assumptions are wrong, seemingly low probability events can occur. In the context of FiveThirtyEight’s stripped-down forecast page, Fivey is a reminder that there’s more than meets the eye. 

But uncertainty that a forecaster can’t quantify is just that: unpredictable. Neither Nate Silver nor the Economist team are in a position to say how far off their forecasts may be as a result of the quality of their assumptions. 

It’s safe to say that these changes to uncertainty communication tell us more about the forecasters and their goals than the future. As readers, we would do well to think twice before taking the numbers, or the messages, at face value. Instead, we should seek information on the model assumptions, and consider what incentives forecasters may have to emphasize (or de-emphasize) the uncertainty. We may never have a crystal ball, but by acknowledging the intricate dance of fact and belief that is uncertainty communication, we can become smarter consumers. 

Jessica Hullman is an associate professor of computer science and journalism at Northwestern University. Follower her on Twitter @JessicaHullman.