Mellman: Complexity makes errors more likely

Two weeks ago, a dinner guest intimately familiar with biohazards explained the irony of the white, full-body suits healthcare workers treating Ebola have been wearing on our TV screens.

Studies have shown the gear to be quite effective but that few people really understand how to take them off without contaminating themselves. Putting them on is no big deal; they were safe then. After use though, the suits are contaminated. That’s of course the whole point: contaminate the suits, instead of the person inside.

{mosads}But whether it’s the massive feeling of relief resulting from escaping the patient’s presence or just lack of training, he told us, people consistently fail to observe the complex protocol required to safely remove the contaminated suits, potentially exposing them to the very virus they are working so hard to avoid.  

A couple of days later, we learned about the Dallas hospital worker who contracted Ebola, possibly by failing to properly doff the biohazard suit. I was impressed with my guest’s perspicacity, but that’s not really my point. 

Advertisers will spend nearly $70 billion dollars on television commercials this year. Most of the pricing for those spots is based on program ratings generated by Nielsen, which admitted last Friday that a software glitch had caused small but meaningful errors in their ratings since March. To its credit, Nielsen admitted the problem publicly and fixed it. Unfortunately, the Dallas healthcare worker cannot get a “do-over,” though we hope for a complete recovery.

Long-Term Capital Management was founded by one of Wall Street’s most successful bond traders and two professors who won the Nobel Prize for the mathematical strategy undergirding the company’s approach. In its first year, it returned 21 percent to investors, 43 percent in the second year and 41 percent in the third year. The year after receiving the Nobel, the firm lost $4.6 billion in fewer than four months, precipitating a crisis that led to intervention and liquidation by the Federal Reserve.

Interesting, you may say, but what has this have to do with us?

Think complexity.

The more complex any process is, the more vulnerable it is to error, and over the years, polling has become much more complex. The rise of mobile phones, low turnouts, greater minority participation and a host of other factors have pushed quality pollsters toward ever more sophisticated and complex methods. 

And with all that complexity comes with a greater possibility of error. 

Earlier this month, New England College released a poll showing New Hampshire Rep. Annie Kuster leading challenger Marilinda Garcia by just 2 percentage points, down from an 11-point margin in their poll a week earlier. The next day those numbers were “recalled” and new data issued, this time showing Kuster leading by 11 points again. 

New England College never explained exactly what happened, though it did own up to the mistake. But “mistakes were made,” as our Soviet friends used to say, and they are made more likely by the complex protocols through which we put polls these day.

Gallup, long considered the gold standard in polling, got the 2012 presidential election wrong. As the campaign drew to a close, Gallup’s daily tracking poll showed President Obama consistently trailing GOP nominee Mitt Romney. After Obama won by 4 points, Gallup worked hard to figure out what went wrong, hiring a distinguished methodologist, Michael Traugott, to lead the investigation. 

In the end there was no one big problem that made all the difference, but at least four factors in their complex methodology together tarnished Gallup’s reputation. Each of the failures resulted from trying to cope with challenges pollsters now face, but each added a layer of complexity, multiplying the sources of error.

Polling is not a simple process, and its complexity carries within the seeds of other errors. In a couple of weeks we will see just how mistake-prone the process can be. 

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.