The Problems With Polls (nybooks.com) 73
Political polling, once hailed as a revolutionary tool for democracy, is facing a crisis of confidence amid high-profile failures and fundamental critiques. Data scientist G. Elliott Morris, Nate Silver's successor at FiveThirtyEight, has defended polling's relevance in a new book, arguing it remains crucial for revealing public opinion despite challenges like plummeting response rates and rising costs.
But critics, including political scientist Lindsay Rogers and sociologist Leo Bogart, have long questioned polling's ability to capture the complexities of public sentiment, arguing it reduces nuanced political matters to simplistic yes/no questions and potentially records opinions that don't exist outside the survey context. Social media platforms, promising to transform democracy by facilitating constant public feedback, have further complicated the polling landscape. The story adds: Today that product remains overwhelmingly popular: polls saturate election coverage, turn politics into a spectator sport, and provide an illusion of control over complex, unpredictable, and fundamentally fickle social forces. That isn't to say that polls don't have uses beyond entertainment: they can be a great asset to campaigns, helping candidates refine their messages and target their resources; they can provide breakdowns of election results that are far more illuminating than the overall vote count; and they can give us a sense -- a vague and sometimes misleading sense -- of what 300 million people or more think about an issue. But, pace Morris, the time for celebrating polls as a bastion of democracy or as a means of bringing elites closer to voters is surely over. The polling industry continues to boom. Democracy isn't faring quite so well.
Silicon Valley ultimately peddled the same feel-good story about democracy as the polling industry: that the powerful are unresponsive to the wider public because they cannot hear their voices, and if only they could hear them, then of course they would listen and act. The virtue of this diagnosis is that structural inequalities in wealth and power are left intact -- all that matters in democracy is that everyone has a voice, regardless of background. In a very narrow, technical sense, their innovations have made this a reality. But the result is a loud, opinionated, and impotent public sphere, coarsened by social and economic divisions and made all the more disillusioned by the discovery that, in politics, it takes more than a voice to be heard.
But critics, including political scientist Lindsay Rogers and sociologist Leo Bogart, have long questioned polling's ability to capture the complexities of public sentiment, arguing it reduces nuanced political matters to simplistic yes/no questions and potentially records opinions that don't exist outside the survey context. Social media platforms, promising to transform democracy by facilitating constant public feedback, have further complicated the polling landscape. The story adds: Today that product remains overwhelmingly popular: polls saturate election coverage, turn politics into a spectator sport, and provide an illusion of control over complex, unpredictable, and fundamentally fickle social forces. That isn't to say that polls don't have uses beyond entertainment: they can be a great asset to campaigns, helping candidates refine their messages and target their resources; they can provide breakdowns of election results that are far more illuminating than the overall vote count; and they can give us a sense -- a vague and sometimes misleading sense -- of what 300 million people or more think about an issue. But, pace Morris, the time for celebrating polls as a bastion of democracy or as a means of bringing elites closer to voters is surely over. The polling industry continues to boom. Democracy isn't faring quite so well.
Silicon Valley ultimately peddled the same feel-good story about democracy as the polling industry: that the powerful are unresponsive to the wider public because they cannot hear their voices, and if only they could hear them, then of course they would listen and act. The virtue of this diagnosis is that structural inequalities in wealth and power are left intact -- all that matters in democracy is that everyone has a voice, regardless of background. In a very narrow, technical sense, their innovations have made this a reality. But the result is a loud, opinionated, and impotent public sphere, coarsened by social and economic divisions and made all the more disillusioned by the discovery that, in politics, it takes more than a voice to be heard.
Re: (Score:2)
This 100%.
My phone is set that all unknown numbers go to voicemail. I'll never be part of a poll.
Re: (Score:2)
I can't believe you even have voicemail activated, tbh
Re: (Score:2)
Why..don't you want to get messages from friends? And if something important is there, like medical related they'll leave a message that I'd want to get.
I'm surprised to hear someone doesn't use voicemail....
Re: (Score:1)
Re: (Score:2)
Many people refuse to participate in polls and just hang up, and Republicans are more likely to do so than Democrats.
Bias in refuseniks skewed the 2016 pews, predicting Hillary would win, and skewed the 2020 polls, predicting an easy Biden win when it turned out to be a squeaker.
Pollsters try to compensate for the bias, but it's hard to get it correct.
Re: Polls (Score:2)
Re: (Score:2)
Yup.
In the current atmosphere, where expressing conservative values can cause you to lose your livelihood....I doubt many people of that slant will talk to any strangers about their views on politics.
Re: Polls (Score:2)
Re: (Score:2)
Where do you live that being an open conservative can cause you to lose your livelihood? Sounds like a pretty cool place to live.
So you want people who don't agree with your political views to lose their jobs? Wow. Just wow.
Re: (Score:2)
>Many people refuse to participate in polls and just hang up, and Republicans are more likely to do so than Democrats.
I thought it was the other way around? I do know that in the U.S., there are far more people who identify as Democrats over Republicans, and as a result a poll that tried for an equal amount of Democrats and Republicans in the sample pool is considered a biased, "bad" poll. It gets more nuanced than that as well, because Democrats are less likely to actually get out and vote, so their the
Re: (Score:2)
Exactly. Any modern polls have to deal with incredible selection bias issues. And that means that any poll numbers require extensive data manipulation or extrapolation based on the small numbers of folks who do reply, leaving themselves open to criticism about methodology, massaging data, etc.
I honestly believe that anything less than hooking random unwilling people up to lie detectors is mostly useless guesswork, or will have a variance of +/-10%, at which point they're useless results anyway.
Re: (Score:2)
If it were that simple, polls would have shown Trump winning in 2016.
Re: Polls (Score:2)
Re: Polls (Score:2)
Polls predict states, states determine the electoral college, and the electoral college determines the winner. (Winning the popular vote is as meaningful as being the tallest candidate, which is to say, it's completely irrelevant.)
A major contributor to polling issues in 2016 was that some Trump supporters chose to lie to the pollsters, telling the pollsters they were voting for Hillary to mess with the campaign.
Re: (Score:2)
"A major contributor to polling issues in 2016 was that some Trump supporters chose to lie to the pollsters, telling the pollsters they were voting for Hillary to mess with the campaign."
Oh, come on. I'm not a never Trump voter and *NOT* a Republican, but this is just not the case. Many chose not to be part of the poll due to their position being perceived as "socially undesirable" and some did chose to "lie", but not for the 'evil' motives you want to attribute to it. Again, to be for Trump was to being
Re: (Score:2)
>> Polls do not predict the electoral college.
They can if they poll each state separately. Those polls are not frequently reported (other than for swing states) because it would take longer to report than a 10 second sound bite and it doesn't serve the media's interest of creating a horse race that's too close to call.
Re: (Score:2)
Re: (Score:2)
Anecdotally...
I have a landline because mobile reception in my house is fickle. I am 54. I hang up on pollsters as soon as I identify that they are one.
If that's typical, Gen-X isn't part of this crap either. We have a secret ballot for a reason, and IMO it's because politics are personal.
The only thing that ever comes into my landline is people who are stuck in the past (doctors, lawyers, banks) and spam. I also use it because my mobile does not accept voicemails, and the landline does.
And that's about how
Re: Polls (Score:2)
Have you heard about WiFi assist? That might allow you to get rid of your landline... it uses your home wifi when cell coverage is lacking.
Re: (Score:2)
I get polls on my cell phone so why does it matter?
Tools of the Corrupt (Score:5, Insightful)
Political polling, once hailed as a revolutionary tool for democracy..
Oh for fucks sake, who’s SELLING that bullshit? The ones profiting off political polls?
Polls are nothing more than an extension of “media”. Always have been. And media is both biased and corrupted by Government. If you struggle to believe that, watch Good Morning, Vietnam again. Then realize that movie had fuck-all to do with a funny radio DJ and everything to do with Government and media censorship.
The lies and clickbait are bad enough, but the censorship issue goes above and beyond. And yeah, that problem is THAT old. I believe today’s political polls and results about as much as I believe the politicians buying and selling them. So should you.
Re: (Score:2)
Indeed. I go into some detail into a comment below about this. Basically there are the scientific polls, which are usually guarded as trade secrets by organizations wealthy enough to conduct them, and then there are the profit-maximizing / ad sales polls that are "farmed" by gaming which method produces the most clicks. For a lot of intuitive reasons, the latter almost always produces statistical ties, because it keeps the
Re: (Score:2)
Re: seriously? (Score:2)
Actually more decisions should be made by a just ruler - firm but fair. ;)
I'm going to suggest you take a moment and reflect on that statement. You really want to be governed by a dictator-like ruler, chosen by 50% plus one voters as "just, but fair"?
Re: (Score:1)
More decisions should be made by policy referendum, not by voting for a political party.
Prop A: Reduce taxes - voters say "yes"
Prop B: Maintain services - voters say "yes"
Prop C: Eliminate the budget deficit - voters say "yes"
Re: (Score:2)
More decisions should be made by policy referendum,
No. If you did this, we would be spending all of our time voting in referenda, and would either be voting out of ignorance, or would be way overloaded in research.
not by voting for a political party.
Agree here; the way party politics works (politicians vote in solidarity with their party or else they are ejected from their party and lose the support needed to win elections) is contraproductive to good government
Actually more decisions should be made by a just ruler - firm but fair. ;)
That begs the question. How do you select "a just ruler"?
Re: (Score:2)
You know what an opinion poll is, right? What's the most important issue on your mind? Who do you have faith in to handle that issue?
I have never heard of a modern poll that asks "Who are you voting for?" It's pointless.
Re: seriously? (Score:2)
It's been asked for decades, along with 'priority' questions...
Re: "revolutionary tool for democracy" my a$$ (Score:2)
Good point, I've never heard the phrase "a strong democracy relies on polling", though people act like it's a valid substitute for actual voting, which democracy does rely on.
I've never had a poll change my opinion/position on a subject - ever. I won't change my opinion on, say, abortion, because 73% of poll respondents have a different opinion than I do, that's not what polls are for.
Polling fails (Score:2)
People have all sorts of reasons for being deceptive, so at the very least you need a massive poll to average that crap out.
People being polled want to influence the results to change perception to influence those easily influenced and will find pollsters who use unethical methods to get the desired polling results.
Different polling methods inherently select subpopulations with average deviances from the general population.
You need to find a way to get a large random sample that isn't accidentally selecting
Re: (Score:1)
* You can get very different answers to essentially the same question depending on factors such as the precise wording of the question or the context in which it is asked. The reasons for the difference are not always obvious even in hindsight.
* All polls have selection bias in one form or another. So do elections, for that matter, but it's impossible to make a poll whose selection biases are a perfect match
Weird (Score:5, Insightful)
I was watching Nate Silver in an interview recently, and he just said it plainly... people who answer polls are "weird." That is, they're less and less representative of the voting population every year. I wonder if you'd be better off sending the questionnaire through the regular mail and having them answer that way. You could argue that people without a fixed address wouldn't be polled, but I think they vote less anyway.
It seems like Twitter and TikTok are where "journalists" turn to but those are the worst. They're performative spaces where people say what they think other people on the platform want to hear, not what they actually think.
Politicians need to know what people are thinking, but there are other ways to find out. If someone from a political party comes door to door, I still talk to them. They can ask me questions and I'll tell them why I'm voting a certain way. Of course if they were recording me, I'd tell them to F-off. What's really damaging democracy is that we have a new wave of people who think freedom of speech is bad. If people are afraid to say what's bothering them out of fear of being ostracized or losing their job, that's what keeps the so called "elites" out of touch with real people.
Seriously, we spent so long under the boot of the church, and being told you can't say that because it's "blasphemy" but these days the list of scientifically proven things you can't say without repercussions is longer and longer. This won't last forever, but I'm afraid of what the backlash against it is going to look like. It's going to be violent.
Re: (Score:2)
Re:Weird (Score:4, Interesting)
What's really damaging democracy is that we have a new wave of people who think freedom of speech is bad.
I'd modify this to say that the new wave think that free speech is Ok - but only for them, and without consequence. They want to say their piece, and not be held accountable for it. And everyone else can just STFU.
Re: Weird (Score:2)
We (voters) don't need polls, campaign staff workers 'need' polls.
Reporters report on polls because the only alternative to reporting on polls would be to instead cover what politicians say, which is incredibly repetitive and boring. By reporting on polling numbers, the election takes on a sporting event-like appeal to readers/viewers.
Re: (Score:2)
Re: (Score:2)
Even limiting ourselves to postwar times, you got the Red Scare and HUAC, J. Edgar Hoover's wire crews listening to everyone who ever expressed an opinion to the left of Genghis Khan, Nixon's "ratfucking" slander machine, violent Southern racists (as always), National Guards deployed because teenagers with hair longer than a millimeter were waving si
The REAL problems (Score:2)
Problem 2 is people lie on purpose to make the candidate they don't want feel safe and spend less money and put in less effort.
Problem 3 is that it's biased towards people who have unlimited time to waste talking on the phone or will take a long political survey for a $5 gift card, aka slightly more Democrats than Republicans.
Problem 4, and this is 99% of the problem, they
Backwards (Score:2)
The fact that polling can be, and is, used to influence results and consensus instead of simply monitoring them as often as not is a pretty damning in and of itself.
Anyone who 'trusts' the results from a poll which can be easily manipulated by any number of factors is a fool, and frankly, that's why they're still used.
Likely voters (Score:2)
Huh. Likely voters? Bascially, boomers. All of us need to fix that. Get out and vote, or they will keep taking and taking until they're gone.
You don't sow, you don't reap.
The problem is perverse incentives, not science. (Score:2)
What that usually looks like is dead-heat horse race elections, with slight emphasis one way or another depending on who
Re: (Score:2)
maybe ... (Score:2)
Political polling, once hailed as a revolutionary tool for democracy, is facing a crisis of confidence
... that's just part of democracy itself facing a crisis of confidence. polls are biased just the same as media, there's no mystery there. at the end money talks and all governments rule overwhelmingly for the elites, the "demos" has no real "kratos" at all. this can only hold while there is hope and trust, meaning money flows, business blooms and everything is good and nice enough to keep the populace happy and content. it's been decades, but we might be facing new challenges and it would seem that such a
2016 really proved the point (Score:2)
No one was admitting to supporting Trump because many Trump voters were afraid to do so for various reasons. My coworkers in a rich, blue region were shocked at how well Trump did. I wasn't. I warned them that most of the USA outside of rich, blue metro regions was not doing well. After watching Bernie Sanders get shated six ways to Sunday, a lot of Trump voters kept their mouths shut.
Partly because a lot of people sympathetic to Sanders actually voted for Trump in the end since pre-sellout Sanders was basi
Re: (Score:2)
Sadly, I think the only effective polling strategy today is to sit down and talk to people and listen. Unfortunately we all feel like the other half are bat-shit crazy stupid morons without a functioning brain cell.
It pains me when smart people make dumb arguments with conviction. It pains me that people hold vile opinions of others based solely on political party. But, I get the divide. It has been around for a long time simmering. Some current turns baffle and worry me, but this too shall pass.
Re: (Score:2)
It's like the way people think Germany elected Hitler, but that never happened - not even close.
I wouldn't join a any club... (Score:2)
Re: (Score:2)
There is more incentive to be dishonest than there is to be honest. When a pollster, somehow, gets through to me, I'm answering questions based on what feedback I want their candidate to have, not on what the pollster is asking.
Will you vote for Kamala Harris? No, I don't like her position on (x, y, z). In fact I wouldn't vote for trump if I had a gun to my head, I'd definitely ask for the bullet. But the democrat party doesn't get a free pass to assume I'm in their corner and support all of their policies,
Pollsters (Score:2)
The last poll taker who came to my house asked "What's the most important issue to you - jobs, the environment, education, or housing?" "None of those" was my response. "Well, which one is the most important to you?" What the hell kind of poll is that?
Better question (Score:2)
A better question would be why are those the only options?
If you want a poll to reflect opinion, you let the takers state their opinion.
If you want to game the results, you limit the options.
Statistics (Score:2)
Contrasting polls
1) Polls of 3,000 (or even 1,000) persons are a joke. For a state like Pennsylvania: 6,915,283 likely voters, 9,090,962 registered, 10,353,548 eligible. Statistical noise. And that's assuming a random sample (polls are "Self-Selecting Populations"). See those error bars? Double them.
2) Poll of 21,000 out of 1.2M Teamsters is statisticly significant. Both in absolute sense (21,000 people) and relative sense (1.75 %).
Yeah, let's blame polls (Score:2)
We should definitely blame polls for their failure to identify winners in statistically tied, winner-take-all races, where a few states will end up dictating the total outcome.
The real problem is the format of the elections. We should be using ranked choice voting and run-offs. The winners should never really be a surprise to us, and their capacity to make sweeping changes without popular support should thus be limited by their ability to remain in power with a less faithful electorate.
other electoral systems [Re:Yeah, let's blame ...] (Score:2)
The real problem is the format of the elections. We should be using ranked choice voting and run-offs.
This is insightful.
The current winner-take-all elections drive us to binary choice polarization.
Agree, ranked choice would be better. I'm a fan of approval voting [electionscience.org], another balloting system that solves the electoral process devolving into a choice between only two alternatives, but ranked choice would be better than what we have.
(Approval voting has one advantage that vote counting is simple, and that no changes are needed to the voting technology. All you need to do is remove the current constraint "Vote f
Bad Options (Score:2)
Polls with loaded questions are just sales tools
Polls with leading questions are to skew the results
People that answer political polls are usually people that want to tell you how to live
All polls without a Cowboy Neal option are not worth answering
Blame phone call spam (Score:2)
Who is paying? (Score:2)
A good poll won't just ask your opinion, they will ask how firm that opinion is and then they will ask a lot of demograp
Misuse (Score:2)
Polls can still be very useful, the issue is the misuse of many of them. Many polls now are built for misuse and outrage. For example, a poll asking people in the country is headed in the right direction makes for great headlines, but doesn't actually tell anyone anything. Do they think it is headed in the wrong direction because it is headed to the left or right? Too many polls are built for headlines instead of for actual gathering data.
The political parties have actual polls (Score:2)
The slop guy's like Nate silver pass around isn't very useful for anything. You might just kind of barely get an idea of a general trend and if the public really really hates a politician, like they do with say JD Vance, you'll know that. But the internal polling from the parties is what matters.
The downside to it is it's extr