Did the VP debate start a trend?

I don’t put a lot of stock into poll numbers.  Instead I look at trends.  Let me explain what I mean.

The hardest part of polling is finding a sample that is representative of the population.  Thirty years ago when every family had only one phone and no answering machine, randomly dialing a household was quite different from today.  In 2012 most everyone over 13 has their own cellphone, and only 9% of polling phone calls result in a response.  The huge changes in how we use our phones induces significant sample error. 

One way of accounting for error is to do what many of the poll aggregators like Real Clear Politics and Nate Silver do.  They lump many polls together to come up with an average response.  But since most pollers have the same problems reaching those who screen their calls, averaging a bunch of polls all containing the same source of error doesn’t make the error go away.  One way of avoiding this error is to look at the same poll over a long period of time.  The numbers themselves might be off, but as long as the same flawed methodology doesn’t change, you should still be able to pick up trends.

A second source of sample error depends on the day of the week when polls are conducted.  I learned when I settled in the South, for example, that many religously active Protestants aren’t home Wednesday evenings because they’re at church instead.  Each day of the week results in a different kind of sample that pollsters try to compensate for by weighting responses.  But weighting induces yet another source of error.

That’s why I like the Gallup rolling average poll.  Because it is a seven-day average, days of the week are not a factor in the poll.  Several other polling groups use three-day moving averages which, while more responsive to changes, are biased depending on the day they are released.  A poll released Friday morning for example, was conducted on the previous three weekdays, while a three-day poll released on Monday samples the population over the weekend.  Even with the same methodology in place, those are two very different samples likely yielding different results.  With a seven-day average poll every day of the week is represented once and only once and that cancels out the bias based on the day of the week

I say all this as prelude to a discussion of the last few days of the Gallup seven-day rolling average poll of the 2012 Presidential race.  After four straight days of a Romney 49 to Obama 47 race, the last three days of polling have seen consecutive one-point gains for Mitt Romney.  On Tuesday Gallup released results indicating that Romney’s lead among likely voters had broadened to 50-46.  Yeterday the new poll release shows an even wider lead of 51-45.  Today the gap widened again to 52-45.

That means that Tuesday’s poll which concluded Monday night, added those polled Monday the 15th of October while it dropped off those polled the previous Monday the 8th.  Since that was the only calculation difference between that poll and the Gallup poll released the previous day, that means that because the poll is an average across seven days, that Mitt Romney polled about seven points higher on the 15th than he did on the 8th while Barack Obama lost seven points when you compare just those two Monday nights.

Now, take that with a grain of salt.  This is a poll that queries a little over 300 people a day.  With such a small sample you have a wide margin of error and are bound to get a few bogus days of results.  However, keep in mind one of the things I always told those I briefed when I presented the results of my own polling that I’ve done for the Army, “Once is an aberration, twice is a coincidence, but three times is a trend.” 

We’re up to a trend on that scale as both yesterday and today Gallup showed the same seven-point jump for Mitt Romney over the previous week’s polls. As for the President, his numbers have fallen by one point in two of the three previous days.  That translates to about a 4 or 5 point drop in Obama’s support over the past week.

I don’t know where the race really stands:  whether it’s Obama up by a couple or Romney up by a few.  But what I do know is that no matter where we started, an 11 or 12 point swing is a rout.

Finally, if the change holds, what could have caused it?  Given the timing I would guess that it was the Vice Presidential debate last Thursday night.  But if that were the case, you would think that we would have seen a change in poll numbers beginning about the 13th of October since that would have been the first release of the poll that included results from after the debate.  However, I don’t think that opinions always change instantly.  It often takes time for things to sink in.  If it was the debate that was the trigger, it probably also took Jon Stewart’s mocking and SNL’s skit, as well as discussions with family and friends, to drive home the point that Joe Biden behaved like an ass.  If that was the case, then don’t expect Barack Obama’s performance at the latest debate to make much of an impact.

Bottom line: it’s possible that the Gallup poll had three consecutive days of outliers and that the poll numbers will shift back to where they were.  However, if this is the beginning of a trend, then for Barack Obama, this is the beginning of the end.

MORE:

Thanks to Glenn for the link.  While you’re here you might want to read about Why the pollsters have been wrong.

Mary Katherine Ham links to a Marquette poll of Wisconsin that indicates a ten-point shift in Mitt Romney’s favor since October 3.  I am especially skeptical of state polling numbers, as most state polls are junk.  However, if there is any truth to the theory that heavily contested swing states are less susceptible to swings because of months of advertising saturation, then further large changes in marginally competitive states like this poll indicates, would be corroborating evidence of a major opinion shift.

Jen Rubin agrees:  “In the Gallup 7-day tracking poll Romney went ahead by 7 points, suggesting that the vice presidential debate (Oct. 11) was, if anything, a positive for the GOP ticket.”

*Some caveats:  I am assuming that each night Gallup polls roughly the same number of people and that their methodology hasn’t changed.  I would also add that, because of rounding, the actual swing may be as few as 8 points or as many as 15.  Even at the low end, that’s a disaster for Team Obama.

Full disclosure: I have a background in polling via work I have done in Iraq where I was a client of Gallup.  Someday when the full program is declassified, I’ll be happy to tell you more. 

Share this post: