Domain-Specific Calibration Development
Calibration develops from domain-specific feedback loops that provide rapid, unambiguous outcome information after predictions, and does not transfer automatically across domains.
This axiom establishes that calibration—the alignment between confidence and accuracy—is not a general trait but a domain-specific skill built through particular feedback structures. Without rapid, unambiguous outcome feedback following predictions, calibration cannot develop naturally, and calibration achieved in one domain does not transfer to others absent similar feedback conditions.
Research on expert calibration demonstrates that weather forecasters develop exceptional calibration (confidence closely matches accuracy) while physicians' calibration remains poor. The critical difference is feedback structure: forecasters receive immediate, objective outcome information after each prediction, while medical diagnosis feedback is delayed, ambiguous, and often absent. Studies attempting to train general calibration skills show limited cross-domain transfer, suggesting calibration is bound to specific feedback ecologies rather than being a transferable metacognitive skill.
This axiom is foundational for the curriculum's approach to developing realistic self-assessment. It explains why calibration training must be domain-specific and feedback-intensive, why confidence assessments are unreliable in domains with poor feedback, and why apparent overconfidence may reflect feedback poverty rather than personality traits. The non-transfer finding is particularly important—it means that being well-calibrated as a poker player provides no calibration advantage when making business decisions.
Practically, this suggests that organizations should engineer feedback systems that provide rapid, unambiguous outcome information, that confidence judgments should be weighted by the quality of historical feedback in that domain, and that general calibration training is likely ineffective. The axiom also implies that new domains require deliberate calibration building through structured prediction-feedback cycles rather than assuming existing calibration will transfer.
Source Lessons
Calibration is domain-specific
Being well-calibrated in one area does not transfer automatically to others.
Well-calibrated perception is a competitive advantage
The ability to see clearly — not optimistically, not pessimistically, but accurately — is rarer and more valuable than most technical skills. Calibrated perception compounds into better decisions, and better decisions compound into better outcomes at every timescale.