Posts Tagged ‘ polls ’

Polls deserve their own set of questions

Colorfully presented polls are the go-to-move for newspapers or researchers looking to boost readership. They even have entertainment value for some. You know those USA Today poll results presented in the lower left-hand corner of the paper? My Dad made guessing the results of the question into a family “game” at the dinner table , much to the pain of my siblings and I.

Whether or  not they provide the basis for Family Feud style competitions, polls add an instant degree of interest and credibility to any report. It’s no surprise that as my class hits overdrive mode on multiple research papers, we’re scrambling to find surveys relevent to our topics, with teachers encouraging us to display the results in creative ways.

But the most snazzy data visualizations (new media speak for what most others call info-charts) won’t cover up a faulty survey. Readings for my Public Opinion class have revealed countless examples of manipulative polls where the methodology or the presentation of the results was highly misleading. Here’s five key questions to ask of every poll as we vet out which make good additions to our own reports.

  1. How were the respondents picked? Those who have taken statistics (which should be a required class in my opinion) know that a few thousand can be a remarkably accurately sample for the views of millions. But it has to be a representative sample. If a poll’s respondents were self selecting like all those online surveys, the results won’t necessarily reflect the entire population. Same is true for a survey that only finds respondents in a certain place, or during a certain time of day, or by only using names in the phone book.
  2. How were the questions worded? Think of the difference between asking about support for “rescuing the auto industry” vs “bailing out a mismanaged corporation.” Very subtle changes in word choice can influence opinions. One’s man’s “government spending” is another’s “public investment.”
  3. Do the respondents have any idea what they’re talking about? The average American knowns little to nothing about foreign affairs or complex domestic policies, but polls come out on these subjects nonetheless asking for opinions. How can we expect a survey to be accurate if it doesn’t screen its respondents for knowledge on the issue?
  4. Is there a “don’t care” choice? Even if you’re knowledgeable about, say, education reform, that doesn’t mean you feel strongly one way or the other about the issue. If a poll presses people into yes or no answers without the choice to opt out, then non-attitudes skew the results.
  5. Who conducted the poll? Political campaigns and private corporations can conduct unbiased polls, and they often do so for internal use. But they have zero incentive to release results unfavorable to their cause. It’s better to stick with credible news organizations, whose self-interest in publishing polls is more about accuracy than  persuasion.