Schiavo polls created opinions

The public pollsters weighed in to the Terri Schiavo controversy heavily, and the results were a mixed bag, with plenty of disappointing performances. Polling on a “new” or emerging issue is always challenging because there are no precedent-setting questions to replicate or questionnaire templates to follow. Looking over the polling on Schiavo, I was particularly struck by one problem.

The public pollsters weighed in to the Terri Schiavo controversy heavily, and the results were a mixed bag, with plenty of disappointing performances.

Polling on a “new” or emerging issue is always challenging because there are no precedent-setting questions to replicate or questionnaire templates to follow. Looking over the polling on Schiavo, I was particularly struck by one problem.

Like journalists, pollsters should be careful to report stories that exist rather than creating stories that didn’t exist prior to their involvement. The premise of polling is that we are measuring attitudes that predate an interview.

There are solid methodological reasons for this. Past research shows that opinions “made up” during an interview are not likely to be stable over time. When respondents are hearing details about a matter for the first time, the pollster’s description is most of the information they possess. The potential for biasing an uninformed respondent is enormous. If subsequent information the respondent sees or hears about that matter is different in nuance and tone, as it is likely to be, then opinions are likely to shift.

In the case of the Schiavo polls, we would have to assume that many of the detailed opinions being measured were invented during the polls themselves. Consequently, many of these opinions were probably biased and short-lived. Polls by Time, ABC News and Gallup bolster this conclusion. All those polls show that, before their interviews, few Americans were following the Schiavo case closely enough to have formed detailed and solid opinions.

The Time poll was conducted March 22-24, late in the chain of events, and still only 33 percent of adults said they were following the case of Schiavo “very closely.” The ABC News poll, conducted a few days before, found that only 16 percent had been following it very closely. A Gallup poll conducted about the same time found that just 27 percent were “very closely” following “the news about Terri Schiavo.”

At the opposite end of the attentiveness spectrum, anywhere from 25 to 44 percent of Americans polled told the pollsters outright that they weren’t following the story. Yet these same uninformed respondents were asked batteries of detailed questions about Schiavo and the controversy surrounding her, many of which would arguably have required some knowledge to answer. Most other polls didn’t even bother to find out whether respondents knew anything or not. They simply started firing questions. The result was dubious data.

As a service to the public, Time, Gallup and ABC News should report their poll results according to two categories of respondents: those who reported being informed about the matter before their interview and those who were uninformed. That would help us understand whether the uninformed voters were being biased by question wordings. If, for example, voters whose familiarity with the Schiavo case was informed mostly by the poll’s questions are found to be more positive toward removal of her feeding tube, then we would have to assume that the wording was leading them to that conclusion.

In the limited space of this column, I cannot possibly discuss all the biased and leading questions that were asked to these largely uninformed Americans, but suffice it to say there were many.

I can point out a few omissions from the surveys that constitute a form of hidden bias. For example, in the eight national public polls I examined, none ever mentioned controversies surrounding Michael Schiavo, whose claim that his wife would have wanted to die were pivotal.

Most important, none mentioned his living with another woman and fathering two children while his wife lay ill. Surely this would have been relevant to some Americans in forming an opinion of his credibility regarding Terri’s wishes.

None of the polls mentioned that some physicians were more optimistic about Terri Schiavo’s prognosis. These issues might have changed hearts and minds. But now we’ll never know.

Hill is director of Hill Research Consultants, a Texas-based firm that has polled for GOP candidates and causes since 1988.