The commenting section closed way too early on Sunday’s opinion piece in the Times on the problems with polling, actually the focus was on political polls.
Before I get to how media measurement plays into polling, what I wanted to say to Cliff Zukin at the Times was about the USE of polls by those who create, distribute and consume them.
I grew up in market research and conducted hundreds of polls and surveys for campaign staffers across the country. The firms I worked for also ran countless surveys for commercial brands and products. The purpose was always aligned with internal marketing objectives – not as a communication vehicle to a target audience. I feel that later has been exasperated by changes within the media and how polters and and their clients interact with the new media landscape.
The Purpose of Polling: Internal vs. External Drivers
From my experience, most surveys designed for marketing use were intentionally biased. We we’re manipulating results, but surveys and polls included questions and question flow that tended to generate the results marketing was looking for. On the other hand, surveys to guide strategy were carefully crafted to minimize errors. When the data was used internally you wanted to know the good and the bad. Sometimes the good was used in marketing materials, but typically the data was used to formulate product strategy, messaging and go-to-market programs.
Polls as Entertainment
Some of the commentary on the NY Times piece summed it up nicely: polls destined for the media are really another form of entertainment. Yes, they do sway voter behavior as more detailed academic studies have shown, but polls 18 months ahead of an election do little to guide strategy then to peak audience attention to the subject(s) in the poll. As a campaign manager I want to “publish” polls showing my candidate is in the lead as it raises his/her visibility and creates more brand awareness for the candidate, i.e. brand building 101.
As a data junky, I love polling data in the media. As a practicing market researcher, I see and understand the questionable methodology that may be at play. I also respect the amazing behind the scenes effort that goes into pulling off a national poll in the current environment of online and offline human behavior.
With polling response rates at an all-time low (now at 8 percent) researchers are struggling to understand their markets. What’s going on in the mind and eventually behaviors of a voter or customers is becoming more difficult and expensive. New online approaches have taken stabs at reducing the cost component, but apparently at the expense of accuracy. An inaccurate poll is ok if you understand the level of inaccuracy (margin of error). Every poll sucks, even the best. But some suck less because they were well-designed during a time when the target population lent itself to the mechanism of random sampling. Polls and survey exist solely on the grounds of sampling, otherwise you’re conducting a census. And when it comes to a voting census, the only accurate reading takes place on November 2nd.
The Role of Media Measurement
The bottom line on media measurement and (not versus) opinion research is it helps clients look further upstream. With the assumption that our opinions and resulting behavior (voting, product consideration, brand selection, etc.) are shaped by “something”, that something is frequently the media or discussions surrounding media reported events. Even social media inputs reportedly originate 80% of the time from news-based coverage. Social media helps shape the discussion around a topic or trend, personalize it with references and anecdotes from people you may know and propel it along the social continuum (viral) to other like-minded individuals.
While opinion research is getting more costly and less accurate, media measurement is becoming more affordable, focused and accurate. It certainly does not replace surveys and polling. And the filtering and categorization aspects inherent in surveys are simply not available in media research. But the ability to gauge multiple media channels across every manner of topic, trend, brand, organization and influencer provides critical insight into the upstream environment shaping downstream opinion formation. As we’ve always championed, media measurement is just one tool in an overall arsenal of analytic solution. With declining response rates, increasing costs and accuracy concerns, the analytic needs of marketers are evolving to include more media-based research tools.