Let me begin by saying that whomever calls it closest doesn’t necessarily win. Sure, I trumpet Rasmussen’s 2008 prediction, but that is simply to demonstrate to those who think that he is a partisan hack that he has a pretty decent track record and cannot be discounted for partisan leanings without being guilty of partisanship yourself.
Before we can answer who grades this election right, we have to first revisit a little basic statistics. Let’s first assume that we have populations large enough as to approximate infinity (we do). Theoretically then, if we want to release polls within a 90% confidence interval (most are 90 or 95 percent, but often they don’t say, so I assume the lower percent), then we need a representative sample of about 1,200 voters to get a margine of error of about 3%. That means that if you gather a 100 different representative samples of 1,200 respondents each, on average, 90 of them will give you the right answer to within a range of about +/- 3 points.
Whoever then gets closest, even if they do everything right in terms of gathering their sample, is as much a game of chance as skill. If the final result is a 50-50 ties, Acme Polling’s November 6th estimate of a 2-point Obama lead is not necessarily worse than Pollco’s prediction that same day of a tie. Let’s look at those two polling companies and see who better tracked the ultimate result.
So who did the better job polling this race? If you go by final result, you would say Pollco because their last poll predicted the ultimate result of a tied 50-50 race. However, if you look at each poll over the last ten days, you see that Pollco average an Obama lead of 6 points while Acme Polling’s average was a tie. It just so happened that on the final day of polling Pollco got a lucky break with their sample, while Acme Polling got a sample that gave them a result within their margin of error. It just happened to be the one-day-in-ten when Pollco was outside their four-point margin of error. However, Pollco’s methodology overall was clearly biased in Obama’s favor. (In statistics ”bias” is a neutral term that has nothing to do with ideology.)
When it comes to determining who did the best job of estimating today’s election, we’re going to have to look at not just the overall result, but their track record as well. If Romney wins by five points, for example, then there is an argument to be made that Gallup did a better job than Rasmussen, because for a longer period of time Gallup was closer to the final result, even though both organizations will have missed it by four. If Obama wins by three, then we’re probably looking at giving the laurels to IBD/TIPP.
Bottom line: you can’t just look at the final number; you have to consider the track record to know which polling company did the best job.