Training Data Bias
AI systems can learn the logic of a dataset’s admission process instead of the logic of the world. If visibility, filtering, or prior selection shaped the training cases, those gates can manufacture correlations the model later treats as reality.
An AI can absorb millions of examples, fit them accurately, and still learn a relationship that does not exist outside its dataset. The culprit is not necessarily bad modelling; it may be the hidden gate that decided which examples were available for training.
E1The filter becomes part of the lesson
Training data is never simply “the world.” Cases first have to become visible, recorded, retained, labelled, and selected. When entry depends on multiple traits, conditioning on that entry can make those traits appear related even when they are independent—a form of Collider Bias associated with Berkson’s Paradox. The model sees only the admitted cases, so it cannot distinguish a pattern produced by the gate from one produced by the underlying population. As AI takes on more decisions, that inherited distortion can be reproduced at scale.
E1Not every correlation is a filtering artefact
A selected dataset can still contain genuine relationships. The warning applies when the route into the data depends on variables involved in the apparent pattern; it is not a licence to dismiss every inconvenient correlation as bias.
Audit admission before accuracy
Before deploying a model, draw the path by which one case enters its training set: who becomes visible, who gets recorded, who receives a label, and who is discarded. Then test the model on cases gathered through a different selection path. If the relationship weakens or reverses, the model may have learned the gate.
Episodes that teach this
-
Why Hot Guys are Jerks and Cute Girls Are Dumb? Berkson's Paradox
· explained at 13:09
9,292 views
The episode warns that AI will make more decisions and that its training data is likely to contain similar selection effects, so it may suffer from Berkson's Paradox.