Things Go South When You Trust DATA Too Much - Goodhart's Law - FutureIQ

4,228 views • Mar 22, 2023

Concepts in this episode

Cobra Effect mechanism

The Cobra Effect occurs when a proxy reward becomes easier to produce than the outcome it is supposed to achieve. Once people can manufacture the rewarded signal, the incentive starts feeding the original problem.
Goodhart's Law core concept

Goodhart's Law begins when a measurement stops being a window onto reality and becomes the object of reward. Once the score carries consequences, people reshape their behavior around the proxy, making it progressively less trustworthy.
Human-in-the-Loop Judgment principle

Data can discipline judgment without replacing it. Reliable data-driven systems preserve a role for experienced humans to interpret context and override outputs that misrepresent the situation.
Perverse Incentives mechanism

Perverse incentives arise when a system rewards behavior that undermines its larger goal. They endure because each participant captures an immediate, legible gain while the resulting harm is delayed, dispersed, or difficult to assign.
Paired Indicators principle

Pair every important target metric with a second metric that measures the damage caused by pursuing it too aggressively. The pair makes the trade-off visible and harder to game.
McNamara Fallacy principle

The McNamara Fallacy is mistaking what can be counted for everything that matters. Quantitative data improves decisions only when interpreted alongside context, judgment, and relevant lived expertise.

Description

Goodhart's law suggests that when the measure becomes the primary target, it ceases to become a good measure. Here is Navin & Shrikant explaining Goodhart's law with different examples and proving how data can be manipulated and can cause more adverse effects than intended. Learn about the Cobra effect, Mcnamara's fallacy and other examples of Goodhart's laws in this simple, fun and short video. Hope you enjoyed FutureIQ by Navin Kabra and Shrikant Joshi. Do hit us up on Twitter: @ngkabra http://twitter.com/ngkabra @shrikant https://twitter.com/shrikant Listen to The FutureIQ Podcast on the podcast provider of your choice: https://tapthe.link/FutureIQRSS Watch other episode: Learn about the law that helps you detect fraud: https://youtu.be/JHA2QpJKGN4 Links: Cobra effect: https://tapthe.link/CobraEffect Keith Rabois' paired indicators: https://tapthe.link/KRPairedIndicators Chapters: Chapters: 00:00 Introduction 00:26 The cobra effect 03:04 Example 2 04:10 Applications of Goodhart's law 04:58 Example 3 05:35 Is data bad? 06:31 Example 4 08:37 Measurements vs Gut instincts 09:53 Example 5 11:00 Example 6 12:20 Conclusion #thefutureiq #cobraeffect

Transcript

Subscriber transcript

Subscribe to @TheFutureIQ, then sign in with Google to unlock full transcripts and transcript search.