Define 3 measurable success criteria before any tool trial, set evaluation at 14-30 days — shorter misses friction, longer activates sunk cost
When evaluating new tools, define three specific measurable criteria before beginning trial and set evaluation period between 14-30 days, as shorter periods miss friction and longer periods activate sunk cost bias.
Why This Is a Rule
Tool evaluation without pre-defined criteria devolves into subjective impression: "It feels nice" or "I think I like it better." These impressions are dominated by novelty bias (new tools feel exciting), feature dazzle (impressive features that may not match your needs), and narrative bias (you construct a story about why this tool is perfect). Pre-defined measurable criteria anchor the evaluation in observable outcomes rather than feelings.
Three criteria (not one, not ten) forces prioritization: you must identify the three most important things the tool needs to do. This prevents both under-specification ("Does it work?") and over-specification (a 20-item checklist that no tool passes). The 14-30 day window balances thoroughness against sunk cost: under 14 days misses friction that emerges only with repeated use (the import flow that's awkward every time, the search that's slightly too slow for daily use), while over 30 days creates sunk cost bias ("I've invested a month in this tool, so I should keep using it regardless of the evaluation").
When This Fires
- Before starting any tool trial (Require a trial period with pre-defined success metrics and a scheduled exit date before any new tool enters your stack permanently)
- When previous tool evaluations ended with "I'm not sure if it's better" because there were no objective criteria
- When Pick a tool with 5 minimum requirements, select the first that meets them, commit for 90 days — satisfice, don't maximize's 5-requirement satisficing needs refinement for the evaluation phase
- Complements Require a trial period with pre-defined success metrics and a scheduled exit date before any new tool enters your stack permanently (trial period requirement) with the specific evaluation design
Common Failure Mode
Post-hoc criteria: starting a trial and then deciding what "success" means based on what the tool happens to do well. This guarantees the tool "passes" because the criteria were reverse-engineered from its strengths. Pre-defined criteria prevent this by setting the bar before the tool has a chance to shift it.
The Protocol
(1) Before installing the trial tool, write exactly 3 measurable success criteria: "Capture from phone takes under 10 seconds," "Daily review load under 15 minutes," "Can find any note within 30 seconds of searching." (2) Set an evaluation end date: 14-30 days from trial start. Mark it on your calendar. (3) During the trial, track performance against the 3 criteria. Not impressions — measurements. (4) On the evaluation date, score each criterion: met or not met. 3/3 → adopt. 2/3 → adopt if the missed criterion isn't critical. 1/3 or 0/3 → reject. (5) The evaluation date is a hard deadline: decide on that day. "One more week" is sunk cost bias talking.