Outlier Trimming
Extreme estimates can pull an average away from the crowd’s central signal. Trimming clearly implausible values can improve aggregation, provided the rule is set consistently rather than chosen to produce a preferred answer.
In a crowd-estimation experiment, the result improved only after two answers were removed. A large group did not automatically protect its average: a tiny number of extreme values could still bend the apparent consensus.
E1How a few answers overpower the many
An arithmetic mean gives every value leverage in proportion to its distance from the rest. Most independent overestimates and underestimates may cancel—the engine behind wisdom-of-the-crowds—while one implausibly large or small answer exerts disproportionate pull. Outlier trimming limits that leverage so the aggregate better represents the cluster rather than its most extreme member.
E1Where it shows up
The two-answer correction
Removing two conspicuous outliers from the experiment shows that aggregation has two separate jobs: collect diverse judgments, then prevent extreme values from dominating their combination. The same safeguard can also improve a crowd-within estimate when one of your independently generated answers is plainly implausible.
E1Trimming can become disguised cherry-picking
An unusual answer is not necessarily an error; it may contain information the majority missed. Trimming also cannot repair correlated-error or selection-bias: if the whole sample leans the same way, removing its extremes may merely produce a tidier version of the same bias. The method is defensible only when “outlier” is defined independently of the result you want.
Set the exclusion rule before seeing the winner
Tomorrow, when combining estimates, calculate the result twice—once with every value and once after applying a predeclared plausibility rule. If trimming materially changes the answer, report both and explain exactly which values were excluded and why.
Episodes that teach this
-
The Wisdom of the Crowd Effect: Theory, Experiment, Examples and Explanation
· explained at 1:07
2,169 views
"we got rid of two outliers... really outliers can mess up things"