Showing posts with label elections. Show all posts
Showing posts with label elections. Show all posts

Political pollsters are pretending they know what's happening. They don't.


This week I have to explain a little statistics, because the political news media have failed so badly.

If you’re like me, you’re probably exhausted with all the so-called news stories about the latest U.S. presidential polls. After each poll, which seems to be a daily occurrence as the election draws near, news outlets report the results on their front pages with breathless excitement. The race is changing! Harris is pulling ahead! Trump is catching up! Swing states are swingier than ever!

To cite just one example, CNN just reported that “Polls show Harris’ numbers in Pennsylvania have shifted over the past month.” This story included a video featuring polling expert Harry Enten, who stated “there has never been a race this close in the polling since 1972!” OMG! (The “OMG” is from me, not Enten.)

Okay, now let’s get to reality (with a small dose of statistics).

A poll is nothing more than a tiny sample of voters’ opinions. A poll might ask, say, 1000 people who they plan to vote for, and then report the results. If you ask a truly representative sample of people, a poll can give you a pretty good idea of how the candidates stand.

The problem is, this is really hard to do accurately, and it’s become much harder since everyone switched to cell phones. Decades ago, pollsters would phone people, and those people would answer the phone. Not any more: many people won’t answer a call from a number they don’t know, and people who do answer might have a bias towards one party or the other.

So pollsters compile some numbers and then adjust them (more on that below), and then report the results with a “margin of error,” which goes something like this. Suppose that a poll finds Trump leading in Nebraska by 18%, with a margin of error of 4%. That means that he might be leading by anywhere from 14% to 22% – and that’s just one pollster’s guess. He might be leading by 30%, or even losing, if you ask a different pollster.

But a 4% margin means that if you run the poll again and again, the results will swing back and forth, randomly, in a pretty wide range.

That’s exactly what’s happening in Pennsylvania. The voters are split, but every poll shows a slightly different result, because that’s what random sampling does. It’s not news!

So let me give you some actual data. Let’s look at Pennsylvania because it seems to be the closest Presidential race, according to the polls. Here are the actual margins of victory from the past 9 elections - real numbers, not polls:

  • 2020: Biden (D) won by 1.2%
  • 2016: Trump (R) by 0.7%
  • 2012: Obama (D) by 5.4%
  • 2008: Obama (D) by 10.3%
  • 2004: GW Bush (R) by 2.5%
  • 2000: Gore (D) by 4.2%
  • 1996: Clinton (D) by 9.2%
  • 1992: Clinton (D) by 9.1%
  • 1988: GHW Bush (R) by 2.3%

Clearly, Pennsylvania has been closely divided in recent years. Suppose the race this year will eventually be won, by either candidate, by less than 2%, as happened in 2016 and 2020. Then what would we expect polls to show us? Given their typical 4% margin of error, we’d expect polls to show a race that flips back and forth from one poll to another, even if no one is changing their mind.

And that, I argue, is just what we’re seeing. The polls aren’t accurate enough, statistically speaking, to tell us anything other than “we don’t know who will win Pennsylvania.” But the media reports each one as if it’s a revelation.

Now back to those “adjustments” that pollsters make. In 2016 and 2020, the polls were off by quite a lot. In 2016, as everyone knows, pollsters were highly confident that Hillary Clinton would win, and they were wrong. They were almost as confident that Joe Biden would win in 2020, and they were right–but the race was closer than they predicted, and their estimates were once again off. In the 2022 midterm elections, though, they were wrong again, but in the opposite direction, and Democrats did better than forecast.

Did they over-correct after 2020, and is that why they predicted much bigger Republican gains in 2022? And have they fixed that now, so that polls this year will be spot on? Who knows?

The thing is, it’s very hard to figure out who will actually show up to vote, and who might answer the phone when a pollster calls, and whether they’re even telling the truth. So pollsters make statistical adjustments based on past experience, weighing some voters more than others. In general, they don’t tell the public precisely how they do this.

Are these adjustments accurate? Well, here’s the kicker: we won’t know until after the election! But one thing is almost certainly true: the changing polls are overstating the number of people who are changing their mind.

So I have a suggestion to the media: stop reporting every poll as if it’s news. Instead, tell us where the candidates stand on issues that really matter: support for Ukraine, support for Israel, health care policy, immigration, respect for the rule of law, stuff like that. I know, crazy stuff, right?

I realize that my plea will fall on deaf ears. It’s so much easier for The Washington Post, CNN, The New York Times, Fox News, and others to write about and talk about polls, and pretend these are actual news. It’s also lazy.

The problem with our democracy isn't gerrymandering. It's integers

(This week, in the wake of the recent U.S. elections, I take a detour from my usual topics to apply a little math to our election system.)

As everyone knows, the U.S. Congress has grown increasingly un-representative. We have states where the population is evenly split among Democrats and Republications, but where–thanks to partisan gerrymandering–the number of House members is grossly skewed in favor of one party. Even without gerrymandering, voters for the losing side in many House districts feel, justifiably, that they have no representation in Congress.

The solution is surprisingly simple. We can save it with a little math. It's not even that complicated. The problem, as I explain below, is integers.

The New York Times just published an argument that the House of Representatives is too small. They point out that the House grew every decade until 1911, when its size was frozen at 435. The Times says that after 100 years of population growth, "America needs a bigger House."

Fair enough, but how do we fix it? The Times argues, oddly enough, that the right number is 593 representatives. Why? Because 593 is the cube root of the total U.S. population. Curiously, many other democracies follow a cube root rule, first described in this 1972 paper by Rein Taagepera. The legislature of Denmark, for instance, has 179 representatives for a population of 5.77 million, and 179 cubed is 5,375,339. Canada has 37 million people and 333 members of their House of Commons, a near-perfect example of this rule, if you ignore their 105 senators.

The first thing to point out is that the NY Times got the math a little bit wrong. The current U.S. population is 329 million, for which the cube root is 690. So if we keep the Senate at 100 members, then we need 590 Representatives in the House, not 593. But the Canadian model doesn't count their senate, so perhaps we need 690 Representatives. But that's a small quibble.

The real problem, though, is that expanding the House by 35% won't address the fundamental problems of our democracy. The Times observed, correctly, that a single representative can't stay in touch with 750,000 people. Increasing the size of the House to 593 will reduce that number to 550,000, which will hardly help. The framers of the Constitution wanted one representative for every 30,000 people, by the way, but that would yield a ridiculously large House today.

The real solution is to get rid of our reliance on integers. Let me explain.

The root of our problem is that each Congressional district elects just one person, in a winner-take-all election where you only need to win by one vote. This means that the losers end up with a Representative who simply doesn't represent them. This means that, in a close election, 49.9% of the voters can be effectively disenfranchised. Even in lopsided victories, where 70% of the voters support the winner, the remaining 30% are stuck with someone who doesn't represent them.

The solution: elect TWO representatives from each Congressional district, and award them each a fractional vote in Congress. Each of the top two vote-getters would have a Congressional vote that is proportional to the number of voters who supported them. Thus if a district elects a Democrat (D) with 55% of the vote, and the losing Republican (R) gets 45%, both of them go to Congress, and D gets 0.55 votes while R gets 0.45 votes.

This will double the size of the House, to 830 members. It will also completely fix partisan gerrymandering. Here's why: imagine a state that is 50-50 Democrat and Republican, but that has packed one district so that 80% of its voters are Republican, allowing it to create three majority-Democratic districts that are 60-40 in favor of D's. Under the current system, that state has 3 Democrats and 1 Republican in Congress. (We have many states that look just like this under our current system.)

Under my new system, our hypothetical state would send 4 D's and 4 R's to Congress. The R from the "packed" district would get 0.8 votes, and the R's from the other three districts would get 0.4 votes each. The entire state delegation would therefore have 0.8 + 0.4 + 0.4 + 0.4 = 2 Republican votes, and 0.2 + 0.6 + 0.6 + 0.6 = 2 Democratic votes, accurately reflecting the overall population of the state.

Gerrymandering is nearly impossible in this system. Packing voters into one district would simply increase the voting power of the majority member for that district, while reducing the voting power of other members of the same party by a corresponding amount.

My system is perfectly legal, and Congress could create it with a simple bill, just as they increased the size of the House in the past. No Constitutional amendment is necessary.

What if more than two people are running for a House seat, as is often the case? We could divide the single House vote proportionally among the top two vote-getters, ignoring the third parties. (States could also use ranked-choice voting to re-apportion the votes of the losing candidates.) A nice side effect is that "protest" votes for third parties wouldn't have such a devastating effect on either of the top candidates. What if only one person ran for a seat? Easy: he or she would get a full vote in Congress rather than a fractional vote. What if the top two vote-getters were from the same party? No problem there, they would both go to Congress, and their party would get a full vote from that district.

Of course, this would make counting votes in the House a bit more complicated. The majority and minority whips wouldn't be able to simply count integers; instead, they'd have to add up the fractional votes of their 435 members. But why should we limit ourselves to a voting system that only uses first grade math? In the U.S., fractions and decimals are covered by the fourth grade. I think Congress can handle that.

There. I've now fixed our democracy. Time to get back to science.