Mark Mellman: Misreading Senate forecasts

When I was in graduate school, deconstructionism was the intellectual rage with my humanities oriented friends. One of deconstruction’s central tenets is that “all reading is misreading.”

I heard it all the time, but frankly, it never meant much to me until I started seeing reactions to the plethora of models now forecasting the results of Senate elections.

{mosads}How are people misreading those results?

First, there is over-interpretation. When HuffPost Pollster says Republicans have a 51 percent chance of taking control of the Senate after this election, or fivethirtyeight.com says it has a 59 percent probability, many people interpret that as meaning, “Republicans are going to take control of the Senate.” That is not at all what these forecasts say.

Such comments turn a probabilistic forecast into a categorical one. It’s transforming a 51 percent or 59 percent chance into a 100 percent (or maybe a 98 percent) chance. If you bet it was going to rain every time there was a 51 percent chance of precipitation, you might come out ahead over time, but you would lose a lot of money.

Others exchange the numbers for words. “Republicans,” they say, “are likely to take over the Senate.” Well, that depends on what “likely” means to you — and research tells us it means different things to different people. For some people, likely is akin to 90 percent; for others, it could be 60 percent. Thinking about the Senate forecasts this way results in a loss of information and precision.

What these numbers may mean is that it is a bit more likely that the GOP wins control of the Senate this cycle than that Democrats keep it.

But consider the classic illustration of probability: coin flipping. If you flipped a coin a thousand times, it should come out heads about 500 times and tails 500 times. If you weren’t actually counting, you couldn’t possibly notice the difference between that 50-50 chance and, say, 510 heads and 490 tails — equivalent to a 51 percent probability. Which is to say, the difference is imperceptible. You probably wouldn’t even notice 590 heads and 410 tails as being particularly off the 50-50 mark. 

When describing what these probabilities mean above, I used the word “may” quite deliberately. What probabilities mean when one talks about repeated flips of a coin is pretty clear. What they mean with respect to discrete events, like whether it rains tomorrow or not, or whether a horse or a candidate wins a race, has been subject to great and unresolved debate.

Frequentists mean just what I said above — if you flipped a fair coin 1,000 times and counted the results, heads would come up 50 percent of the time. Bayesians mean that, based on your knowledge about the world, you have a 50 percent certainty that the next flip will be heads. 

What the election forecasting models are really saying is that, in circumstances like these, we expect that 51 percent or 59 percent of the time, Republicans will win the Senate. Presumably most forecasters have gone the next step and examined past forecasts to find that outcomes they judged 51 percent or 59 percent likely actually occurred about 51 percent or 59 percent of the time and, just as importantly, did not happen 49 percent or 41 percent of the time.

If things judged 59 percent likely to happen occurred 100 percent of the time, the forecasts would be wrong.

Which takes us back to the first point: a 51 percent or 59 percent chance of Republicans taking the Senate is not the same as saying the GOP will, or even is very likely to, prevail. 

When people aren’t busy over-interpreting the models, they are under-appreciating their meaning and import. More about that in a future column.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.