Pew Study Finds No Easy Fix for Bogus Respondents in Online Opt-In Polls
Pew Research Center tested three methods for removing fraudulent respondents from online opt-in polls: trap questions, CloudResearch's Sentry prescreening, and matching to a voter file. Analyzing 11,114 respondents, they found that while purging bogus cases generally reduces error, no approach is foolproof. Voter file matching actually increased error by removing valid respondents, while trap questions and prescreening performed similarly. All three methods modestly overstated Democratic support in the 2024 election, as bogus respondents tended to claim they voted for the winner, Donald Trump.
Matching an opt-in sample to voter files slightly increased error by removing mostly good respondents (e.g., those who simply declined to give their name or address).
- 0points
And here I was thinking this was proved beyond any reasonable doubt when 4chan users spoofed the Time Magazine person of the year poll of 2009 to spell out "MARBLECAKE, ALSO THE GAME" by the initials of the top winners:
https://techcrunch.com/2009/04/27/time-magazine-throws-up-it...
Or when Kim Jong Un won in 2012, spelling out "KJUGASCHAMBER":
https://observer.com/2012/12/4chan-successfully-votes-kim-jo...
- noduerme
The bit about 10% of respondents claiming they had visited the International Space Station was funny. Reminded me of this nugget[0], which for people unfamiliar with the subject doesn't mean that 20% of Jews believe in Jesus... it means that 20% of the respondents to the poll were either deranged or confused, and self-identity as Jewish for no clear reason. It's analogous to "1 in 5 vegans say they eat meat."
https://www.timesofisrael.com/jews-for-jesus-poll-15-of-jewi...
- tokioyoyo
From time to time we've battled multiple ideas around polling and tech within my friend circles, because there really isn't "reliable" way of knowing "what people want" when it comes to government's decision makings. It's such a hard "reach and incentives" problem, that asking for an opinion, might change the opinion itself as well, depended on the questioning. That being said, it would be cool to see real opinions of people on general matters, rather than heavily sampled versions of it.
- rsktaker
How about just not doing online opt-in polls
Nobody enjoys that experience; I personally am a "bogus" respondent.
- nilamo
I have no sympathy for polls. Y'all sending SMS unsolicited is in violation of the TCPA, and I'm not going to respond to an organization that has so little respect for me.