What does it mean that overconfidence is the default calibration error?

Your brain does not fail randomly. It fails in a specific, measurable, predictable direction: too much confidence. Across decades of research, in every population tested, the dominant calibration error is overconfidence — believing you know more than you do, that your estimates are more precise than they are, and that your performance exceeds what it actually achieves.

Example: A product manager asks her team of eight engineers to estimate how long a migration will take. She asks each person to give a range they are 90% confident will contain the actual duration — meaning they believe there is only a 10% chance the real answer falls outside their range. Every engineer gives a range. The migration takes fourteen weeks. Six of the eight ranges do not include fourteen weeks. Their 90% confidence intervals captured the true value only 25% of the time. This is not an unusual result. In software estimation research, when professionals provide 90% confidence intervals, the true value falls inside those intervals only 60-70% of the time (Jorgensen, 2004). The engineers were not lying or lazy. They were exhibiting the single most robust finding in the psychology of judgment: humans are systematically, predictably, measurably overconfident.

Try this: Run a ten-question calibration test on yourself right now. For each question, estimate a numerical range you are 90% confident contains the true answer. Use questions with verifiable answers: the population of Brazil, the height of the Eiffel Tower in meters, the year the first iPhone was released, the distance from Earth to Mars in kilometers at closest approach, the number of bones in the adult human body, the GDP of Japan in trillions of USD, the boiling point of ethanol in Celsius, the length of the Nile River in kilometers, the number of symphonies Beethoven composed, the speed of sound in meters per second at sea level. Write your ranges before looking anything up. Then verify each answer. Count how many of your 90% confidence intervals contained the true value. If you are well-calibrated, nine out of ten should. If you are typical, four to six will. The gap between your target (90%) and your hit rate is your overconfidence score. That number is the size of the problem this lesson addresses.