Mark Mellman: There is danger at the margin

Pollsters and pundits spend lots of time studying margins — how far ahead or behind a given candidate is relative to his or her opponent. Focusing on margins is a dangerous game, however, and one of the reasons polls appeared off the mark this past year.

Surveys are designed to elicit point estimates, the level of support for each candidate. The margin has no particular statistical standing. It is merely the difference in the level of support garnered by each candidate. And therein lies the problem.

{mosads}First, let’s dispense with the more prosaic, but nonetheless important, statistical issue. When polls cite a statistical margin of error (which is always a function only of sample size), it is for the level of each candidate’s support. It is not for the margin. 

Say Candidate A has a 5-point lead, 48 percent to 43 percent, in a poll with a 4-point margin of error. Many would think A’s lead might be as large as 9 (5+4), or as little as 1 (5-4). Not really; the margin of error applies to each candidates’ vote share.

Correctly (but over-simply) applied to the point estimates for the candidates, A’s support could be between 52 percent (48+4) and 44 percent (48-4), and B’s between 39 percent (43-4) and 47 percent (43+4). Thus, (oversimplifying greatly) A’s margin could be between a 13-point lead (52 to 39) and a 3-point deficit (44 to 47).

In short, as a statistical matter, the margin of error on the margin is substantially larger than the margin of error for the poll (which is to say, for the estimate of each candidate’s support provided by the poll).

There is, however, a deeper, conceptual problem with focusing on margins. They ignore the undecideds.

When I first started polling, the insiders’ secret was that margins were irrelevant in incumbent races. It was, more or less, a “what you see is what you get” world for incumbents. 

An incumbent at 45 percent at the end of a campaign was likely to end up with close to 45 percent even if leading his or her opponent by 5 or 10 points. Undecideds broke overwhelmingly to challengers.

In the very first House race I polled, I was able to tell my client that we had a good shot of winning even though we were more than 30 points behind two weeks out, because the incumbent was well under 50 percent.  (And win we did.)

Alas, as I have documented here over the years, this rule no longer applies. Undecideds do not necessarily break to challengers, and they haven’t for some time.

But how undecideds break does matter, and when one focuses on margins, their role is obscured.

This cycle, undecideds broke quite heavily toward Republican candidates. 

By and large, polls pegged Democrats’ vote rather precisely. Everyone was shocked by Sen. Mark Warner’s 1-point victory in Virginia, but the average of late polls were within seven-tenths of a point of his actual vote total. 

In all but five cases, the final poll averages in Senate races were within 3.5 percentage points of Democrats’ actual vote totals. They were just three-tenths of a point off for Michelle Nunn and 1 point off for Alison Lundergan Grimes.

So what happened?

While Democratic Senate candidates picked up an average of 1.9 points, Republicans gained 6. David Perdue added 5.2 points to his poll total, Bill Cassidy 7.6, and Ed Gillespie 9.6 points.

Of course, no one predicted in advance that the undecideds would all break that way, but figuring out before the fact where they will end up is the trick.

Polls are built to assess each candidate’s level of support, not the margin. Predicting the margin requires a deeper understanding of how undecideds will break, and pollsters would be wise to develop that knowledge.

I know I keep trying. 

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the minority leader of the Senate and the Democratic whip in the House.