It’s almost embarrassing. Polls are supposed to accurately measure attitudes, but on any given day they seem to meander randomly, leaving readers with an inexplicably inconsistent portrait of the political landscape. A few weeks ago, I focused on bouncing presidential approval ratings, but the problem is also evident in match-ups.
Gallup began publishing its daily tracking results 10 days ago, claiming President Obama was 2 points behind Mitt Romney, which jumped to 5 points the very next day. Three days after recording another 5-point Romney margin, Gallup suddenly had the president ahead by 3.
{mosads}Indeed, within the space of four days last week, polls showed President Obama anywhere from 9 points ahead (CNN/ORC) to 2 points behind (Fox News). Really?
Noting that the variation resists easy explanation, Huffington Post’s always astute Mark Blumenthal offered two possibilities worth exploring — random variation and unsettled preferences.
The significance of sampling variation shouldn’t be underestimated. Most surveys peg the vote for each candidate within a narrow band. As Obama and Romney traded places in the Gallup data, both candidates’ support varied within just a 4-point range.
But don’t overestimate sampling error, either — the probability that Obama is up 9 and down 2 at the same time, simply as a result of statistical error, is rather remote.
Voters’ unsettled preferences are also important. When pollsters push the truly undecided to support a candidate, they are asking respondents to tell more than they know, thereby eliciting noise.
Voters don’t have to make up their minds until Election Day, so why should they make decisions in April merely to satisfy the cravings of pollsters and junkies? Exit polls tell us that even in the highly charged atmosphere of 2008,
10 percent did not make up their minds until the week of the election and a quarter decided in October or November. Those self-reports shouldn’t be wholly trusted, but with undecideds in recent polls running from 5 percent to 9 percent, something appears out of kilter.
It might well be. When George Bishop and colleagues at the University of Cincinnati asked people whether they favored or opposed repealing the Public Affairs Act of 1975, over a third offered an opinion about nonexistent legislation — an opinion they could not have had, but were called upon to make up, because the question asked them to. When the question explicitly offered the opportunity to be uncertain, false opinions nosedived to the low single digits.
Presidential elections are real, but most pollsters push respondents hard to offer an opinion, whether they’ve got one or not. Non-attitudes can create the appearance of volatility.
Other insightful analysts, like Ron Brownstein and Alan Abramowitz, focus on varying minority composition of the samples as another potential culprit.
Building on their discussion from a different direction, all these polls show both Obama and Romney capturing about 90 percent of the vote from their respective partisans. That partisan unanimity gives two factors outsized influence on the overall result — the percentage of Democrats and Republicans picked up by each poll and the disposition of the small group of independents.
Suffice it to say, the partisan distribution in these polls varies meaningfully. Some give Democrats as much as an 8-point advantage in party ID, while others suggest a Republican plurality. With a nearly one-for-one translation of party into vote, that will make a difference.
Second, the relatively few independents make for large margins of error, and widely varying results, among those independents. Gallup’s Romney-leaning poll had him winning independents by 6 points, while the CNN/ORC poll that looked strong for Obama gave the president a 6-point margin with independents.
Non-attitudes, the partisan composition of the sample and random variation within the relatively small independent bloc can move poll results in very different directions, making it no less frustrating for poll watchers, but perhaps somewhat more explicable.
Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the Majority Leader of the Senate and the Democratic Whip in the House.