ROC Curve
An ROC curve maps the price of catching more true cases: as sensitivity rises, false positives usually rise too. It lets you choose a threshold whose errors fit the real costs instead of chasing impossible perfection.
A detector can catch more of what you fear and still become less useful. Turn up its sensitivity and genuine cases stop slipping through—but innocent cases are swept in with them.
E1Every threshold buys one error with another
A detection system assigns cases a score, then draws a cutoff. Lowering that cutoff labels more cases positive: true-positive detection improves, but false alarms multiply. Raising it reverses both effects. Plotting these paired outcomes across possible cutoffs produces the ROC curve—the geometry of sensitivity versus selectivity.
The curve does not select the correct cutoff for you. That requires the cost of each mistake: missing a dangerous case versus burdening a harmless one. Like an optimal-error-rate, the defensible operating point may deliberately tolerate some failures because eliminating them would cause greater damage elsewhere.
E1The curve cannot price the consequences
An ROC curve describes a trade-off, not the stakes behind it. Two thresholds can look like nearby points on the graph while producing radically different human or financial consequences. Nor does the curve promise a neat optimum like a revenue-maximization-curve; the choice still depends on which error matters more in the actual setting.
E1Cost the two mistakes before moving the cutoff
For one decision system you use tomorrow, write down the concrete consequence of a false negative and a false positive. Rank which is costlier, then move the threshold in the direction that reduces that error—and explicitly record the increase in the other one that you are accepting.
Episodes that teach this
-
Cost Benefit Analysis Real Life Examples - Sensitivity vs Selectivity
· explained at 2:52
1,168 views
The transcript says sensitivity and selectivity are a trade-off, and 'when you plot that on a graph you get something called The ROC curve.'