Mellman should refocus

Mark Mellman, my Democrat pollster foil here in the pages of The Hill, is an accomplished and deservedly respected researcher. But he took a wrong turn last week in attacking focus groups.

Mark Mellman, my Democrat pollster foil here in the pages of The Hill, is an accomplished and deservedly respected researcher. But he took a wrong turn last week in attacking focus groups. Recall his statements about the use of focus groups for testing of political TV ads:

“Focus groups are about the worst technique ever devised for this purpose.”

“People are very poor reporters of their own decision making process.”

“People’s well-documented tendency to conform to group pressure also threatens the validity of focus group results.  . . . We see the impact of conformity in focus groups all the time.”

Re-reading his remarks, I came to the conclusion that it’s not focus groups that Mark Mellman loathes; it’s the focus group participants that bug him. Somehow voters aren’t insightful enough to test the very ads that are designed to influence them.

Anyone who has moderated or observed a focus group sympathizes with Melman’s thinking. Sometimes the people who participate in groups seem to be the moral equivalent of down-and-out drifters who donate blood for money. “Here, take a pint and give me $75.” They force us to reexamine whether we want their opinions, much less their plasma.

Yet there are ways to have better focus groups that get around issues raised by Mark. Political focus groups can be done better.

First, we can do a better job recruiting focus group participants. Rather than simply being satisfied with a panel of dull participants that roughly conforms to the demographic profile of the target market, do more. Also screen participants in advance for their conversation and analytical skills. Few political researchers do this, but it’s not uncommon for skilled market researchers to do so.

Second, we can moderate better. Good moderators can ask probing questions that reach deeply into the values and beliefs that underlie and give form to opinions about a particular ad. Rather than simply asking a person to rate an ad, try to go beneath the surface of their first (or even last) opinion about an ad to see what values are driving their ratings.

Third, we can use questioning techniques that get around the self-reporting problems Mark makes reference to. Voters are sometimes incapable of discussing their own views, but I have found that they are more articulate when I ask them how their neighbor will respond to an ad. Once they get clear of the fear of revealing too much about themselves, they will dig deep to tell you about opinions that are likely to surround a particular ad.

What troubled me most about Mark’s attack on focus groups is his denigration of the effects of conformity. He’s doubtless seen focus groups where voters start agreeing with the more vocal or charismatic participants in the group. Well, instead of seeing this as a weakness of focus groups, it must be recognized as a strength. Public opinion formation is a social process. We do talk about ads over the water cooler with co-workers or over the backyard fence with neighbors. These conversations influence our opinions. It’s inconvenient for the researcher that assumes that all opinion formation is private, but it’s reality. If voters care so little about their opinion of an ad that they will change their views to conform to those of others, shouldn’t we want to know that?

Mark insinuates that there are better ways to test ads, but he fails to mention any weaknesses of the alternatives. I suppose he’s recommending Internet-based testing, where broadband PC users sit all alone at their desks and “rate” ads they see in a small 2 or 3 inch window. This is altogether better? So much better that we should dismiss focus groups to the trash bin? Let’s not we jump to rash conclusions.

Hill is director of Hill Research Consultants, a Texas-based firm that has polled for GOP candidates and causes since 1988.