Classify evidence by diagnostic value before updating your beliefs
When new evidence arrives, classify it by diagnostic value before updating—ask whether you'd see this evidence regardless of belief truth versus only if belief were true/false.
Why This Is a Rule
Not all evidence is equally informative. A customer cancellation after your price increase might feel like strong evidence that the price is too high — but customers also cancel for dozens of unrelated reasons. You'd see cancellations whether the pricing was right or wrong. That makes the cancellation low-diagnostic: it doesn't distinguish between the world where your belief is true and the world where it's false.
High-diagnostic evidence is evidence you'd expect to see in one world but not the other. If the customer cited pricing specifically in their exit survey, that's high-diagnostic — you'd see this comment if pricing was wrong and wouldn't see it if pricing was right. The evidence discriminates between the two possibilities.
This rule prevents the common error of updating beliefs on any evidence that feels relevant. Before updating, classify: would you see this evidence regardless of whether your belief is true or false? If yes, the evidence is low-diagnostic and should barely move your estimate. Only high-diagnostic evidence — evidence that discriminates between possibilities — warrants significant belief updates.
When This Fires
- Interpreting customer behavior, market signals, or team dynamics
- Evaluating whether new data supports or undermines a hypothesis
- Deciding how much to update a project estimate based on recent results
- Any time new information arrives and you're about to change your plan
Common Failure Mode
Counting confirmations without checking diagnostic value. Every piece of evidence that's consistent with your belief feels like support — but most evidence is consistent with multiple hypotheses. Seeing a long line at a restaurant is consistent with "the food is great" and also with "it's the only restaurant nearby" and "they're slow." The observation doesn't help you discriminate between these explanations, so it shouldn't move your estimate much.
The Protocol
When new evidence arrives: (1) Ask: "Would I see this evidence if my belief were true AND if my belief were false?" (2) If yes to both → low-diagnostic. Barely update. (3) If you'd see it mainly when your belief is true (or mainly when false) → high-diagnostic. Update significantly. (4) Adjust the size of your update to the diagnostic value, not to the emotional impact of the evidence.