Of the 23 national polls predicting the final popular vote count, eight of the ten worst estimates came from media-sponsored polls, while none of the top-five closest polls were affiliated with a media organization.
I have a theory as to why this is. Consider the news coverage of this presidential election. How much coverage of issues did you see? How much in-depth investigative reporting was there? Not much. Well, except for reports of Sarah Palin’s family. But seriously, what major news organization researched and reported on the collective body of Barack Obama’s votes in the Illinois State Senate? I don’t remember seeing it. Nor had I seen any similar report about John McCain’s quarter-century in Washington.
What we saw instead was a whole lot of horse race coverage. And I think that the reason for that is that it is much cheaper. Hiring a good investigative reporter, and putting him on an expense account (you’d be amazed how much a journalist can charge while he’s out of town) so that he can dig into years of legislative records and talk with former colleagues and lobbyists, is a very expensive endeavor. To do all that in order to produce–if you’re lucky–two stories after two weeks of investigation costs tens of thousands of dollars. Buying into a poll is a lot cheaper and you’re guaranteed a new story every few days. But it’s not reporting.
So how does that explain why the media polls got it so wrong? Simple. You get what you pay for and what the media demanded was quick-result polls that didn’t cost much. They wanted to report every morning the results of a poll concluded the previous night. I’ve done a little bit of polling work before. It is difficult to scrub the data for outliers and check it against detailed stratifications that quickly. To save the media money the polling organizations cut corners. They made assumptions that they then didn’t go back and re-test, they accepted far too many non-responses without adequate follow-ups, and they simply automated the results without installing enough human checks in the system to see if the results made sense. Humans cost money and take time, neither of which was something the media wanted.
In order to cut costs, the media stopped doing their traditional jobs and produced pre-packaged polling material instead. Not only did they get it wrong, their declining circulation numbers show the ultimate result.