A smart friend wondered today why Intrade still expects an Obama win even when the national poll numbers are slipping away from him. It’s a good question that I’ll attempt to answer in two parts. The first part is about why Obama optimism is unfounded. (The second part is here.)
First let’s look at the Cook Partisan Voting Index in the chart below. I’ve arranged each state numerically by PVI from most Democratic Washington DC to GOP stronghold Utah. The third column is that state’s electoral votes. The last column is the running total of EV. (As an aside, since Colorado is the only state with a PVI of zero and its 9 electoral votes include the one that get either candidate to 270, this is why it is highly unlikely that a spread in the popular vote by much more than a point will present a situation where the overall vote winner doesn’t also win at least 270 electoral votes.)
Compare where the last two weeks of the election was fought four years ago with where it is being contested today. Most of the remaining swing states in 2012 are on the Democratic side of Colorado, which serves kind of like a 50-yard line marker. One could quibble at the margin over this list of states. For example, four years ago McCain went into Pennsylvania late in the race when he saw that Ohio was shutting down on him and this year there is some indication that the race is about to extend into Michigan and Minnesota. However, there’s no question that the current race has shifted up the scale of states in the favor of Republicans. Four states where Obama marched strongly into the opponent’s territory four years ago (Arizona, Georgia, Indiana, and Missouri) are on nobody’s list of possible Obama wins this time.
Consider also what the PVI means. It is the amount by which a state is more or less Republican than the national average. If for example, Barack Obama wins nationally by two points, we should expect him to get most states with a PVI of less than two. If current national polls (according to the RCP average) are accurate, then Mitt Romney leads by about one point. That means that if every state follows its partisan leanings, then Iowa and Nevada are on the bubble to flip to the Republican side. That leaves Barack Obama to fight an upstream battle that requires him to keep those two states, plus either Colorado, which is one point more Republican than the spread, or Ohio, which is one point more Republican still.
As an example, imagine that in 2008 Barack Obama had narrowly lost Ohio, and that every state shifted right by 4.6% (the amount by which he secured the Buckeye State). That would have kept Florida, North Carolina, and Indiana in McCain’s column. But still, it wouldn’t have mattered. Obama would have won Colorado and Virginia and a total of 291 electoral votes.
Yes, Ohio probably goes with the ultimate winner, but because the list of remaining contested states has shifted north from four years ago, that means that Mitt Romney does indeed have more paths to victory available to him. In fact, if yesterday’s polls are accurate, Romney has a lead of three points. Sure, Obama may still win Ohio through the brute strength of his ground game there, but it probably won’t help. To win overall, he will also have to hold onto another 61 electoral votes from the 7 Democratic-leaning states that that have a PVI greater than the national margin of 3 points. That’s a lot of territory to defend.
One of the primary sources of Democratic optimism springs from Nate Silver, a former sabremetrician and blogger. Four years ago he built a complex mathematical model that correctly predicted 49 of 50 state outcomes in the presidential race. From his higher perch now at the New York Times, he confidently predicts that Obama has a 73.1% chance of winning and that he is likely to take about 294 electoral votes. Read the comments on blogs both right and left; Democrats wield Silver’s predictions like a crucifix in front of a vampire of contrary poll results. The headline in today’s (London) Telegraph, for example, adoringly calls him “the geeky statistician who is singlehandedly dismantling the myth of Mitt-mentum.”
The first crack in Silver’s statistical prognostications appeared in 2010 when his early projections significantly undercounted Republican gains in the House. Eleven days before the election he predicted that there was a 70% chance that Republicans would gain less than 60 seats. They won 63. That alone should be enough to remind observors that there shouldn’t be so much certitude about Silver’s 70% predictions a week and-a-half away from a vote.
But there’s another problem with Silver’s model; and it’s a problem that a sabremetrician should most studiously attempt to avoid: It is based on the wrong statistic.
Sabremetrics is a portmanteau derived from a
nearly 20 40-year old group known as the Society for American Baseball Research. SABR sits at the intersection of mathematics and sports and its overwhelming desire is the search for the perfect measure of success. These are the guys that found that instead of tracking batting averages or runs batted in, a far more accurate predictor of offensive baseball success is OPS: on-base percentage plus slugging. (Moneyball, the story of Billy Beane’s use of sabremetric-like statistics to create a winning ball club in a small market team, is a great read on this subject.)
Silver’s model relies heavily on one metric: the spread between candidates–and especially on the spread in state level polling. There is a problem with both parts of that and with the model itself.
First, the model. Most good time-series mathematical modeling is validated against past events in order to predict future unknowns. Furthermore, greater weight is given to more recent events when verifying the model. That is usually the smart way to model a problem–except, that is, when the recent event with the greatest weight happens to be an historical outlier. The 2008 presidential election was an outlier. It was the first election since 1952 when there was neither an incumbent president nor a sitting vice president on the ballot. Since it was a contest unencumbered by incumbency, late-breaking undecideds were not predisposed by external factors to break one way or the other. Going into election day, the RCP average showed about a 7-point lead for Obama over McCain and that’s the way it ended up on election day. In other words, late-breakers broke to each side in about the same proportion as the decided portion of the electorate. However, when there is an incumbent on the ballot, it is uncommon for him to get the late-breaking vote. Look at the the chart from 2004 below:
After John Kerry gained points at George W. Bush’s expense during the first debate, and after John Kerry gave some of those points away when he made a stupid third-debate remark about Dick Cheney’s daughter being a lesbian, we see a distinct pattern over the last two weeks of the race. Bush’s numbers are stuck. Meanwhile, the challenger John Kerry saw significant gains from his depths four days after the last debate. In the last two weeks of the 2012 race we should expect to see a similar pattern. Why? Why not. So far we have seen a similar pattern between the Bush-Kerry race and the Obama-Romney race all the way up to this point. Here’s the same chart, but with both the 2004 and 2012 races superimposed. They are almost identical in shape for both incumbent and challenger.
Furthermore, the pattern of this race bears no resemblance at all to what occurred in 2008. Therefore, there is nothing up to this point that would lead us to believe that 2008 is a good predictor for today.
Nate Silver’s model tells you where the race is. (More accurately, his model tells you where the race was, as data is usually about 2-7 days old.) But it doesn’t account for where the race is going. In 2008 that wasn’t a problem as late-breakers broke proportionately. However, two weeks before the 2004 election, Silver’s model would have underestimated the challenger’s gains, just as his model underestimated the gains of Republican House challengers two years ago. His is not a dynamic model that takes into account historical patterns and thus, it is unable to project future results. That doesn’t necessarily make it a bad model, so long as you keep in mind the limitation that he produces a snapshot of the recent past and not a vision of the future. Based on the most recent historical precedent for the 2012 election, Barack Obama is not in good shape when he is already behind a challenger who hasn’t yet seen his late-breaking surge.
But there is another problem with Silver’s model. By relying so heavily on the spread between candidates to predict results, it misses the point that not all spreads are the same. An incumbent with a two-point lead who is sitting at 49 points is in much better shape than an incumbent with the same two-point lead but who is at 45. The incumbent’s level of support, not the spread, is the most important metric in a re-election race. That is because it tells you how safe the incumbent is from the effects of a last-minute surge. On that metric, Barack Obama is not safe at all. Since at least 2010, when the creation of Obamacare led the news, Barack Obama has struggled with his support, only briefly breaking the 50% barrier.
Even more alarmingly for the President, dissatisfaction is both strong and stable while his support has been lukewarm. Rasmussen has polled the level of those who strongly aprove of the President and those who strongly disapprove. It’s not a pretty picture for the President.
Rasmussen shows that for a long time well over 40% of the electorate has strongly disapproved of the President, but only about 30% now strongly approve. Neither the support for the incumbent nor the strength of it appears to be a metric in Nate Silver’s model. Therefore, the over-reliance on the spread as a predictor of success makes for a model that doesn’t account for hard ceilings and soft support. If you were a well-funded challenger, where would you rather be two weeks before an election: 4 points behind an incumbent stuck at 47%, or 2 points behind an incumbent at 49? Even worse for Obama, is that he is stuck at 47 and already behind.
The final problem with Nate Silver’s model is its over-reliance on open-source state polling. Good polling is very expensive. That is why so few organizations do it well. Furthermore, even those who do polling well, don’t do it the same way for all clients. As an Army analyst, I have hired one of the major national polling companies to conduct recurring nationwide polling of a sufficiently large sample size to get statistically meaningful state level results. The cost: $11 million. Sure, that was in Iraq and not the United States, but labor costs there are much less than here. Nonetheless, I bring this up to point out that a good poll of the scope and scale necessary to derive national level results from state level input costs more money than even the largest media organizations can afford. Only the campaigns themselves have the kind of money to do polling right. All the publicly-released polls cut corners in order to do it on a budget. They use small sample sizes, and loose voter screens. Nor do they go back and check sample responses against the answers from a sample of non-responders. That’s why at the state level you get wide variability and large swings in most polls.
The optimism that Obama’s supporters project is unfounded in hard data and historical precedent. Against the numbers, his fans point to GOTV, state polls that buck the national norm, and magical statisticians who assure them that all is well. The only thing missing is the inevitable last minute appeal to the ghost of Harry S. Truman.
Barring an unprecedented shift, Barack Obama is unlikely to win the popular vote. That alone is enough to place him in an electoral disadvantage, worthy of no higher than a 50-50 chance. But still, there they are: Democrats and Intraders and their irrational exuberance.
Tomorrow, Part II will explore the reasons why the reality-based community is having trouble coming to grips with their reality.