Why does agent success metrics fail?

The most common reason agent success metrics fails: Defining metrics that are easy to count rather than meaningful to track. You measure 'number of journal entries per week' instead of 'percentage of entries that surface an actionable insight.' The easy metric gives you a green dashboard while the agent silently underperforms. This is Goodhart's Law applied to your own cognition — the metric becomes the target, and the actual goal drifts out of view.

The fix: Pick one cognitive agent you currently run — a decision-making heuristic, a weekly review process, a reading protocol, anything. Write down three metrics that would tell you whether it is succeeding: one measuring whether it fires at all (reliability), one measuring whether it produces the intended output (effectiveness), and one measuring cost (time, energy, or cognitive load). Be specific enough that someone else could measure them without asking you what you mean.

The underlying principle is straightforward: Every agent needs a clear definition of what success looks like in measurable terms. Without operational metrics, monitoring produces noise instead of signal.