How do I practice agent reliability metrics?

The most direct way to practice agent reliability metrics is through a focused exercise: Select three cognitive agents you rely on regularly — your daily planning agent, your emotional regulation agent during conflict, your focused-work agent, your active-listening agent, or any others you have identified in earlier phases. For each agent, define: (1) The trigger condition — what situation should activate it? (2) The observation window — the past 14 days. (3) The hit count — how many times the trigger occurred and the agent fired correctly. (4) The miss count — how many times the trigger occurred and the agent failed to fire. (5) The false-fire count — how many times the agent fired when the trigger condition was not actually met. Calculate reliability rate (hits / total triggers) and false-fire rate (false fires / non-trigger occasions) for each. You now have six numbers that describe the reliability profile of three core agents. Write them down. This is the beginning of your monitoring dashboard's reliability layer.

Common pitfall: Treating reliability as a binary — the agent either 'works' or 'doesn't work.' This collapses a rich, multi-dimensional signal into a useless bit. An agent with 95% reliability and a 30% false-fire rate has a completely different failure profile than an agent with 70% reliability and a 0% false-fire rate. The first fires almost every time it should but also fires when it should not — a sensitivity-heavy, specificity-poor agent. The second misses nearly a third of its triggers but never fires incorrectly — a conservative, specificity-heavy agent. These two agents need opposite interventions. Treating both as simply 'unreliable' leads you to apply the wrong fix and wonder why nothing improves.

This practice connects to Phase 28 (Agent Monitoring) — building it as a repeatable habit compounds over time.