In practice, the sample size used in a study is usually determined based on the cost, time, or convenience of collecting the data, and the need for it to offer sufficient statistical power.
You see how the elements listed there (cost, time, convenience, and ‘sufficient statistical power’) are more qualitative measurements and not known constants? (I mean, whenever it starts with, “In practice . . .” you know it means “in a perfect system devoid of unknowns”, or in other words “ideally but you’ll see it doesn’t work exactly like that” )
What is the sufficient statistical power for sampling Europe? 0.002%? Two thousandths of a single percent? That greenlights your findings? Okay. I disagree. Polling companies don’t disagree because in this case, as you noted, 20k is an amazing sample size. The cost and time for that - not to mention the convenience! - alone is amazing . . for an opinion poll. No doubt they’re proud, that’s a fine achievement for an opinion poll. Now: did they measure what they set out to measure? I doubt it, but since the methodology given is the single word “online”, I remain skeptical.
And saying “but there’s math in it!” is exactly why I’m skeptical. That effectively means nothing, and it’s used to validate whatever conclusions were presented. “We ran the numbers, and . . ” can mean very specific things, and in some contexts it is good enough to move on to the conclusions. Polls trade on that, but they don’t deserve to.
From your link:
You see how the elements listed there (cost, time, convenience, and ‘sufficient statistical power’) are more qualitative measurements and not known constants? (I mean, whenever it starts with, “In practice . . .” you know it means “in a perfect system devoid of unknowns”, or in other words “ideally but you’ll see it doesn’t work exactly like that” )
What is the sufficient statistical power for sampling Europe? 0.002%? Two thousandths of a single percent? That greenlights your findings? Okay. I disagree. Polling companies don’t disagree because in this case, as you noted, 20k is an amazing sample size. The cost and time for that - not to mention the convenience! - alone is amazing . . for an opinion poll. No doubt they’re proud, that’s a fine achievement for an opinion poll. Now: did they measure what they set out to measure? I doubt it, but since the methodology given is the single word “online”, I remain skeptical.
And saying “but there’s math in it!” is exactly why I’m skeptical. That effectively means nothing, and it’s used to validate whatever conclusions were presented. “We ran the numbers, and . . ” can mean very specific things, and in some contexts it is good enough to move on to the conclusions. Polls trade on that, but they don’t deserve to.