Sensemaker Daily
In a snap poll conducted by CNN after Tuesday’s presidential debate, Kamala Harris’s net favourability rating rose 12 points while former President Trump’s fell by one.
So what? It was one metric, in one poll, conducted by one polling company, at one point in time.
In a media environment overflowing with coverage of new polls and discussion of different models, it’s worth remembering three things:
- Individual polls are distinct from polling models but feed into them.
- Shifting patterns over the past two or three election cycles have decreased polling accuracy and increased scepticism of both polls and models.
- The media’s fixation on national polling is largely misplaced, considering the mechanics of the electoral college.
Polls versus models. Individual polls ask actual people actual questions. Models combine polls to discern and explain trends.
In its most straightforward form, a model looks like a simple average of every poll. In its most complicated, it takes into account other data including things like…
- when each poll was published;
- its sample size;
- the historical accuracy of the pollster;
- how often the polling company releases new polls; and
- whether it includes “likely voters”, “registered voters” or “all adults”.
Pollsters also make choices about how to collect and interpret data, which leads to debate. As Ben Ansell – host of the Tortoise podcast What’s Wrong With Democracy? – wrote this week: “[I]n an election this close, every assumption matters.” (Emphasis his.)
Models are generally more accurate than individual polls. But – blame the news cycle – the release of singular surveys (see the CNN/SRSS poll mentioned above or the New York Times/Siena College poll last week) tend to get more coverage, which can in turn skew public perception of the race.
And… the fact that some publications, like the New York Times, publish both models and their own individual polls doesn’t help news consumers untangle the two.
Moving targets. According to a report from the American Association for Public Opinion Research (AAPOR), national-level presidential polls four years ago were the least accurate they’d been in 40 years.
- The report – essentially an industry mea culpa after failings in 2016 as well – called the underestimation of support for Trump in 2020 (by an average of 3.9 per cent in national polls and 4.3 per cent in state polls) an error of “unusual magnitude”.
- The polling industry has come to a general view that at least some of the inaccuracy in 2016 can be explained by too many (mostly state-level) polls not accounting for education.
- But what caused the abnormally large error in 2020 – and whether it’s been corrected – remains an open question.
Polling models are only as good as the polls that go into them, and the industry’s inability to consistently estimate support for Trump has made both polling and models harder to understand.
Oh, also… polling was off in the opposite direction two years ago. Democrats outperformed predictions in the 2022 midterm elections. Go figure.
538. Ultimately what matters in a presidential election is not who gets more votes overall, but who wins the popular vote in enough states to secure 270 electoral votes.
In this election, there are really only seven states where the outcome could still go either way: Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin. For Harris, there are few paths to victory that don’t involve winning Pennsylvania. For Trump, it’s hard to see how he recaptures the White House without carrying Georgia.
How to read polls. When in doubt…
- look at models and polling averages…
- of – where possible – swing state polls...
- and don’t be surprised if there’s another big error this year.
In what direction? Close your eyes and point.
And bear in mind… On the most recent episode of What’s Wrong With Democracy?, Jim Messina, who ran President Obama’s 2012 re-election campaign, said that he used to threaten to fire his staff if they talked about daily polls.
