Lies, Damn Lies and Opinion Polls – Really?

Scoop Werewold 12 March 2012
“Conservative Young Cautious on Sex Education,” claimed a headline in the Sunday Star Times early in January. According to the paper, “sex education could be too much for conservative Kiwi youths” who, it concluded, hold “conservative views on sex issues”. In the face of recurring moral panics about our over-sexualised, over-sexed drunken youth, it was certainly a surprising claim. Could it be that the nation’s teenagers are all ensconced in their bedrooms doing their maths homework after all?

The Sunday Star Times’s article was based on a poll commissioned by the conservative lobby group Family First, whose media release on the same day had pretty much the same take, bearing the headline: “Teens Conservative on Sex/Abortion Issues – Poll”. “A nationwide poll of 600 young people aged 15-21 has found that they hold conservative values on sex issues,” the release began.

A cursory glance at the survey data actually tells quite a different story – different, at least, depending on the story you want them to tell. What’s more, Family First’s own release provided evidence of less – rather than more – sexual conservatism, with its expressions of concern at New Zealand’s high teen pregnancy and abortion rates, and “out of control” sexually transmitted infections. None of which seemed to disturb the Sunday Star Times in its desire to run with a headline that wasn’t just perkily counterintuitive, but also got to include the three-letter “s” word. Nor did any doubts plague The Dominion Post’s columnist Karl du Fresne, who was heartened that the survey showed teenagers “see through all the fraudulent b…..” about sex education .

The “conservative teens” survey was one of around a dozen opinion polls released by Family First in the past year on a range of hot-button moral issues, many of which, just like the youth survey, attracted some surprisingly uncritical attention by the mainstream media. But of course it’s not just Family First surveys that sometimes get an easy ride. Under the “polling” category on their StatsChat blog, faculty from the University of Auckland’s Statistics Department list numerous examples of poor use of polls on issues ranging from smoking to students fleeing overseas to the media’s own “self-selected web site polls”. “We’ve commented before on the annoying tendency of newspapers to claim that self-selected website polls actually mean something,” Prof. Thomas Lumley wrote in a January post.  “The media usually refers to the results as coming from an ‘unscientific poll’, but a better term would be ‘a bogus poll’”.