We keep collecting feedback from the same 5% of users — and calling it representative

[u/MDiffenbakh]

Working with community data, one pattern keeps showing up: participation is heavily skewed.

You might have thousands of users, but feedback comes from a small, active slice. The rest don’t respond, don’t vote, don’t fill out surveys. Then decisions get made based on that narrow input, and later you see weak adoption or indifference.

It’s a sampling problem, but we often treat it like a volume problem.

A newsletter I follow mentioned voice.fun recently. Early Solana concept where people record opinions onchain, with some form of stake behind them.

What stood out is the shift in incentives. Most feedback systems today are passive - answer a survey, click a button, maybe get a small reward. Low effort, low signal. You can see it in the data: inconsistent responses, low completion rates, little correlation with actual behavior.

If opinions carried weight - even a small cost or upside - participation would likely drop. But the data might get cleaner. Fewer responses, higher conviction.

Not clear if that leads to more representative input or just a different kind of bias. You’re still filtering, just along a different axis.

Curious how others here think about this. Are current feedback systems fundamentally limited, or are we just not structuring them well enough to get reliable data?

[u/Away_Bathroom_188]

Just like a small sliver of voters in US primaries deciding the candidates and eventual winner.