Of polls and megaphones

CNN is a great news organization peopled by wonderful journalists.
Their survey department is also run by fine pollsters. So I really do
hate to pick on them, especially for a fault in which they are hardly
alone.

But, they are the most current example of a news outlet that has created a huge story out of its own poll, even when its results differ from those of other surveys.

{mosads}CNN’s latest poll, which — as far as I know — adheres to high methodological standards, seemed particularly newsworthy precisely because it showed dramatic change in President Obama’s standing with the American public.

CNN’s own Web article highlighted the “plunge” in Obama’s approval rating.

As the piece noted, the president’s approval rating “dropped 8 percentage points over the last month to 45%, the President’s lowest rating in more than 8 months. … And Obama’s disapproval rating soared 9 points to 54% …”

Academics and analysts speculated online and on-air about the reasons for the precipitous decline.

Some evidence seemed to point to the National Security Agency revelations, although a (bare) majority favored the program and only a minority thought the Obama administration had gone too far in “restricting people’s civil liberties.”

Others pointed to the decline in the economy, though consumer confidence had been rising until this month, when it eased off a bit. The story spread like wildfire, with other journalists repeating the meme of the president’s falling approval. It threatens to take on a reality of its own.

But what if it’s not true?

Of course, it is truly the result obtained by CNN/ORC poll. Polls purport to reflect reality, but what if this particular poll does not reflect the underlying reality?

Of course, it’s entirely possible that by the time you read this, other polls will have been released validating CNN/ORC’s findings. But as of today, CNN/ORC is an outlier, as other polls paint a rather different picture.

Gallup’s newly retooled poll (more about that in a future column), conducted after CNN’s but already in the public domain, found a 47 percent plurality approving of Obama’s performance, with 44 percent disapproving.

Thus, while CNN finds Americans disapproving of the president’s performance by a 9-point margin, Gallup says they approve by 3 points. Moreover, unlike CNN, Gallup’s data reveal no negative trend.

Depending on which earlier poll you want to compare to, Gallup would have us believe the president’s rating either stayed steady or actually improved ever so slightly.

Gallup is not alone in offering a different version of reality.  A Time survey that ended the day CNN/ORC’s field work began also showed Americans approving of the president’s performance by a 4-point margin (48 percent to 44 percent).

I am not in the position to say one set of data is correct and the other is incorrect. But I am reasonably sure that CNN poll is not more accurate because it bears the CNN name (nor is Gallup’s more right because it has their brand).

But CNN’s huge megaphone has consequences. Many more see and hear about the CNN numbers than about those produced by Gallup or Time.

I certainly understand CNN’s interest in promoting work bearing its brand — that’s good for business. However, precisely because reality is uncertain and various polls produce somewhat different portraits of that reality, a little modesty is in order.

None of the coverage of this poll that I’ve seen even mentions the differing results in other surveys. Would any reporter simply ignore conflicting sources on other stories?

If a news organization produces results out of sync with those of other reputable pollsters, it seems to me they bear some responsibility to put their own numbers in context.

Plummeting approval is a better story, it just may not be an accurate one — and doesn’t accuracy rank pretty high in the pantheon of journalistic values?

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.