This week I have to explain a little statistics, because the political news media have failed so badly.
If you’re like me, you’re probably exhausted with all the so-called news stories about the latest U.S. presidential polls. After each poll, which seems to be a daily occurrence as the election draws near, news outlets report the results on their front pages with breathless excitement. The race is changing! Harris is pulling ahead! Trump is catching up! Swing states are swingier than ever!
To cite just one example, CNN just reported that “Polls show Harris’ numbers in Pennsylvania have shifted over the past month.” This story included a video featuring polling expert Harry Enten, who stated “there has never been a race this close in the polling since 1972!” OMG! (The “OMG” is from me, not Enten.)
Okay, now let’s get to reality (with a small dose of statistics).
A poll is nothing more than a tiny sample of voters’ opinions. A poll might ask, say, 1000 people who they plan to vote for, and then report the results. If you ask a truly representative sample of people, a poll can give you a pretty good idea of how the candidates stand.
The problem is, this is really hard to do accurately, and it’s become much harder since everyone switched to cell phones. Decades ago, pollsters would phone people, and those people would answer the phone. Not any more: many people won’t answer a call from a number they don’t know, and people who do answer might have a bias towards one party or the other.
So pollsters compile some numbers and then adjust them (more on that below), and then report the results with a “margin of error,” which goes something like this. Suppose that a poll finds Trump leading in Nebraska by 18%, with a margin of error of 4%. That means that he might be leading by anywhere from 14% to 22% – and that’s just one pollster’s guess. He might be leading by 30%, or even losing, if you ask a different pollster.
But a 4% margin means that if you run the poll again and again, the results will swing back and forth, randomly, in a pretty wide range.
That’s exactly what’s happening in Pennsylvania. The voters are split, but every poll shows a slightly different result, because that’s what random sampling does. It’s not news!
So let me give you some actual data. Let’s look at Pennsylvania because it seems to be the closest Presidential race, according to the polls. Here are the actual margins of victory from the past 9 elections - real numbers, not polls:
- 2020: Biden (D) won by 1.2%
- 2016: Trump (R) by 0.7%
- 2012: Obama (D) by 5.4%
- 2008: Obama (D) by 10.3%
- 2004: GW Bush (R) by 2.5%
- 2000: Gore (D) by 4.2%
- 1996: Clinton (D) by 9.2%
- 1992: Clinton (D) by 9.1%
- 1988: GHW Bush (R) by 2.3%
Clearly, Pennsylvania has been closely divided in recent years. Suppose the race this year will eventually be won, by either candidate, by less than 2%, as happened in 2016 and 2020. Then what would we expect polls to show us? Given their typical 4% margin of error, we’d expect polls to show a race that flips back and forth from one poll to another, even if no one is changing their mind.
And that, I argue, is just what we’re seeing. The polls aren’t accurate enough, statistically speaking, to tell us anything other than “we don’t know who will win Pennsylvania.” But the media reports each one as if it’s a revelation.
Now back to those “adjustments” that pollsters make. In 2016 and 2020, the polls were off by quite a lot. In 2016, as everyone knows, pollsters were highly confident that Hillary Clinton would win, and they were wrong. They were almost as confident that Joe Biden would win in 2020, and they were right–but the race was closer than they predicted, and their estimates were once again off. In the 2022 midterm elections, though, they were wrong again, but in the opposite direction, and Democrats did better than forecast.
Did they over-correct after 2020, and is that why they predicted much bigger Republican gains in 2022? And have they fixed that now, so that polls this year will be spot on? Who knows?
The thing is, it’s very hard to figure out who will actually show up to vote, and who might answer the phone when a pollster calls, and whether they’re even telling the truth. So pollsters make statistical adjustments based on past experience, weighing some voters more than others. In general, they don’t tell the public precisely how they do this.
Are these adjustments accurate? Well, here’s the kicker: we won’t know until after the election! But one thing is almost certainly true: the changing polls are overstating the number of people who are changing their mind.
So I have a suggestion to the media: stop reporting every poll as if it’s news. Instead, tell us where the candidates stand on issues that really matter: support for Ukraine, support for Israel, health care policy, immigration, respect for the rule of law, stuff like that. I know, crazy stuff, right?
I realize that my plea will fall on deaf ears. It’s so much easier for The Washington Post, CNN, The New York Times, Fox News, and others to write about and talk about polls, and pretend these are actual news. It’s also lazy.