Words matter in pollsters’ questions

It seems like a simple question: Do Americans now support the Obama administration’s efforts to save our auto industry? The question is of particular moment for Mitt Romney in Michigan’s primary because the scion of an auto-industry CEO penned a 2008 New York Times op-ed headlined “Let Detroit Go Bankrupt.” Not only was his recommendation economically misguided, Romney’s prediction proved fundamentally flawed. If the Bush/Obama proposal went forward, Romney opined, “you can kiss the American automotive industry goodbye.”

Of course, the plan was implemented and its success beyond dispute. A principled critique of its animating philosophy could be offered, but no one can reasonably suggest it failed or that auto companies are about to exit the scene, as Romney inaccurately predicted. 

{mosads}But public opinion is about perception, not reality. And two respected, independent pollsters provide strikingly different assessments of the voters’ verdict. The Pew Research Center headlined its findings “Auto Bailout Now Backed,” while Gallup reported “Americans are more likely to disapprove than approve of the federal government’s bailout of U.S. automakers.” Which is it?

Start with the basic facts. Pew asked, “The government also gave loans to General Motors and Chrysler during this period. Do you think this was mostly good or mostly bad for the economy?” Fifty-six percent (56 percent) replied mostly good, while 38 percent took the opposite view. 

Gallup took another tack, asking, “Now thinking back to one of the major actions taken by the federal government in the last four years, would you say you approve or disapprove of the financial bailout for U.S. automakers that were in danger of failing?” Forty-four percent (44 percent) approved, whereas a bare 51 percent majority disapproved. 

Note that while the results are on both sides of the 50 percent line, neither represents overwhelming consensus. The difference in support proffered by Gallup and Pew respondents is 12 points. That’s far from trivial, but if one poll had shown support at 54 percent and another at 66, I doubt I’d be writing this. 

Nonetheless, it’s enough of a difference to require explanation, and Gallup’s Frank Newport suggests, “One reason for the different responses may be the Pew focus on the impact of the actions on the economy, while we at Gallup asked for a broader approve/disapprove response.” Newport implies Gallup’s is the more politically relevant measure, as “It is certainly possible that an individual may feel that a certain action helps the economy … but be philosophically or practically opposed to that action.”

It is just as likely that by calling attention to the federal government, Gallup is inadvertently measuring a partisan reaction. Modest support for this interpretation arises from the greater partisan polarization evident in Gallup’s responses.

Another difference in the question wording, unremarked upon in Gallup’s analysis, is its use of the word “bailout.” Pew talks about “loans,” technically more accurate and certainly less inflammatory, though “bailout” is clearly common parlance. It should surprise no one that Americans prefer “loans” to “bailouts.” 

Fortunately, Pew adds some clarity in the midst of this morass of confusion, as its item is tracked from a nearly identical question posed in 2009. The percentage believing loans to the autos improved the economy shot up 19 points since then, while the number seeing harm dropped by 16. Whatever the actual level of support for rescuing the auto companies, that 35-point net shift reflects strong positive momentum.

There’s a broader lesson here. The next time someone attempts to interpret public opinion based on responses to a single question, at one point in time, recall this example and be suspicious — seemingly small differences in wording, barely noticeable without thinking through alternatives, could be exerting a substantial impact on the answers.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the Majority Leader of the Senate and the Democratic Whip in the House.