No Easy Fix for Bogus Respondents in Online Opt-In Polls

(pewresearch.org)

49 points | by luu 1 day ago

12 comments

  • 0points 8 hours ago
    And here I was thinking this was proved beyond any reasonable doubt when 4chan users spoofed the Time Magazine person of the year poll of 2009 to spell out "MARBLECAKE, ALSO THE GAME" by the initials of the top winners:

    https://techcrunch.com/2009/04/27/time-magazine-throws-up-it...

    Or when Kim Jong Un won in 2012, spelling out "KJUGASCHAMBER":

    https://observer.com/2012/12/4chan-successfully-votes-kim-jo...

    • hyperpape 3 hours ago
      You're half right, half not.

      This is evidence, but it would be a mistake to think that high profile online contests would have the same dynamics as low profile online polling. 4chan is characterized by an insane level of effort on the specific jokes that they care about. But they don't care about a random poll nearly as much as they care about big stunts like this.

      • nemomarx 3 hours ago
        Of course, but a low profile poll also needs less effort to manipulate, so it scales down for both ends.
    • aniviacat 4 hours ago
      Online polls being easy to manipulate makes it so weird to me that the European Central Bank made an online poll on what the new Euro bank note design should be. (Participants did not have to identify themselves in any way.) I wonder if the results will at all be taken into consideration in the following decision-making process.

      https://www.ecb.europa.eu/press/pr/date/2026/html/ecb.pr2609...

      • account42 2 hours ago
        In that case there was heavy pre-selection und strict rules for submissions even before that. So they were probably fine with any of the presented choices winning and the "poll" is really just PR.
    • rhdunn 5 hours ago
      Or Boaty McBoatface winning the name of a research ship (given to a submarine on the ship).
      • dcminter 3 hours ago
        In fairness I think a lot of people (me included) genuinely thought that would be an entertaining name for the ship, so the polling result wasn't necessarily wrong per se.
        • rhdunn 3 hours ago
          Fair, though it does show how things like that can skew polling results.
          • mitxela 2 hours ago
            i.e. engagement bait
      • alchemism 2 hours ago
        At the time, we ended up with a conference room with the same name, after an office vote.
      • c4rnifex 26 minutes ago
        Those were not bogus respondents, just silly ones.
  • noduerme 5 hours ago
    The bit about 10% of respondents claiming they had visited the International Space Station was funny. Reminded me of this nugget[0], which for people unfamiliar with the subject doesn't mean that 20% of Jews believe in Jesus... it means that 20% of the respondents to the poll were either deranged or confused, and self-identity as Jewish for no clear reason. It's analogous to "1 in 5 vegans say they eat meat."

    https://www.timesofisrael.com/jews-for-jesus-poll-15-of-jewi...

  • tokioyoyo 7 hours ago
    From time to time we've battled multiple ideas around polling and tech within my friend circles, because there really isn't "reliable" way of knowing "what people want" when it comes to government's decision makings. It's such a hard "reach and incentives" problem, that asking for an opinion, might change the opinion itself as well, depended on the questioning. That being said, it would be cool to see real opinions of people on general matters, rather than heavily sampled versions of it.
  • nilamo 1 hour ago
    I have no sympathy for polls. Y'all sending SMS unsolicited is in violation of the TCPA, and I'm not going to respond to an organization that has so little respect for me.
  • Ellis_dev 3 hours ago
    The lure of a few bucks for minimal effort always attracts fraud. Our own survey data is consistently tainted by bots and click-farmers.
    • tgv 3 hours ago
      Also by "straight-liners," as we call them: people who just click items in a straight line to finish asap. And then there are the nonsense respondents. Don't trust poll data.
  • rsktaker 4 hours ago
    How about just not doing online opt-in polls

    Nobody enjoys that experience; I personally am a "bogus" respondent.

    • yorwba 43 minutes ago
      Why do you opt in to the poll then?
  • tokai 1 hour ago
    I cannot find the paper again, but a small group of women giving birth in the US healthcare system do self report as virgins. There is not protecting for misunderstandings or malicious intent, even in person.
  • parineum 8 hours ago
    The easy fix is to not do online polling.
    • hennell 8 hours ago
      You need to though. If you only do phone call polls you're only going to poll people with (landline?) phones who answer (random?) calls and have the time/enthusiasm to respond when you call. That heavily skews demographics.

      A lot of people will only do online polling, and it's by far the easiest way to cover a lot of demographics (need to supplement with other methods as well if you don't want a poll of "internet users" - but that is a far larger and varied group than most other methods would produce)

      • tgv 3 hours ago
        Actual visiting. Has been done. Still seems to be the preferred method in parts of the world with unreliable internet.
      • parineum 2 hours ago
        That's like punching the ground because it hurts to punch the wall.

        Online polling isn't an improvement, it's just a different error.

        • yorwba 38 minutes ago
          Getting a different error is great! Then if you do an online poll and a phone poll, you can combine the results to get a composite poll that's more accurate than either method alone.
    • LateCheckOut 8 hours ago
      [dead]
  • dyauspitr 7 hours ago
    Polls seem completely useless over the past decade or so. I think betting markets are a better indicator of true sentiment now.
    • tancop 5 hours ago
      Betting markets are even less representative. People who trade on politics are usually rich conservative men with a risk taking mindset and more likely to be active crypto users or work in finance. It's a totally different user base from normal polls.
      • gruez 1 hour ago
        The point of markets isn't to recruit a diverse user base, it's to create an incentive structure that punishes bad predictions and rewards good predictions.
  • jodapogo 5 hours ago
    [flagged]
  • darig 9 hours ago
    [dead]