Check if indirect indicators could agree for reasons other than truth — shared sources count as one
When using indirect evidence, assess whether indicators are genuinely independent by checking if they could agree for reasons other than the schema being true—if all evidence shares a common causal source, it counts as single evidence despite multiple data points.
Why This Is a Rule
Triangulation — using multiple lines of evidence to validate a claim — only works when the evidence lines are genuinely independent. Three dashboard metrics all derived from the same data pipeline are not three lines of evidence; they're one line presented three ways. If the pipeline has a bug, all three metrics agree — and their agreement falsely strengthens confidence.
The independence test asks: "Could these indicators agree for reasons other than the schema being true?" If yes, a common causal source might be driving the agreement rather than the schema's accuracy. Customer satisfaction surveys, support ticket volume, and renewal rates might all improve simultaneously — not because customers are genuinely happier, but because a new manager is filtering negative feedback at the intake level.
This is Count only independent evidence lines for confidence — ten sources sharing one origin is one line, not ten (count independent evidence lines) applied specifically to indirect evidence: when you can't observe the thing directly and must rely on indicators, the indicators' independence determines whether convergence is meaningful or illusory.
When This Fires
- When multiple indicators converge and you're about to increase confidence
- During any validation that relies on indirect measures rather than direct observation
- When dashboard metrics all move in the same direction and you want to verify the signal
- Any time "multiple data points agree" is used as evidence for a conclusion
Common Failure Mode
Counting converging indicators without checking their independence: "Three metrics all show improvement!" Check: do all three come from the same data source? Are they computed from overlapping inputs? Could a single upstream change produce all three improvements? If yes, the convergence is one signal dressed as three.
The Protocol
When indirect indicators converge: (1) For each indicator pair, ask: "Could these two agree for a reason other than the schema being true?" (2) Trace the causal chain: what data source does each indicator rely on? (3) If indicators share a causal source → they count as one evidence line regardless of how many there are. (4) Only genuinely independent indicators — different data sources, different methodologies, different causal chains — provide triangulation strength.