Sensitivity vs Selectivity
Detection systems cannot maximize both catching true cases and sparing legitimate ones. Raising sensitivity usually buys fewer misses at the cost of more false positives, so the right threshold depends on which error is more costly.
A fraud system that stopped every fraudulent payment would also stop real customers. Perfect vigilance sounds ideal until the detector starts treating ordinary transactions as crimes.
E1Every threshold chooses an error
A detector must draw a boundary between signal and noise. Move that boundary to catch more true fraud and ambiguous legitimate payments cross it too; move it back to protect legitimate activity and some fraud escapes. Sensitivity measures how many true cases you catch, while selectivity reflects how well you reject cases that do not belong. You cannot simply turn both to one.
The choice is therefore not “accurate or inaccurate” but which kind of mistake the system is built to tolerate. Error Management Theory explains why false alarms can be rational when a missed threat is especially costly. But when alarms become abundant, the signal-to-noise-ratio deteriorates and legitimate activity bears the cost.
E1The trade-off is not a fixed law of performance
This mechanism applies when a system is adjusting a threshold over overlapping cases. Better information may improve the detector rather than merely shift its cutoff. And after filtering cases, selection itself can create an apparent trade-off that was not present in the original population.
E1Price both mistakes before tuning
Before tightening any fraud rule, write down the two resulting errors: fraud that slips through and legitimate transactions that get blocked. Estimate the cost of each, then choose the threshold explicitly. Do not approve “maximum detection” without also naming the false-positive rate you are willing to impose.
E1Episodes that teach this
-
Cost Benefit Analysis Real Life Examples - Sensitivity vs Selectivity
· explained at 2:21
1,168 views
To prevent all fraud, the system would also flag real transactions; 'you can't make selectivity and sensitivity both one.'