Evaluating new tools: 30 minutes to a verdict.
Something launches every week that's 'about to change everything.' Most won't. Some will. The skill isn't predicting — it's a cheap, repeatable evaluation that gets you to adopt/watch/skip in half an hour, so the firehose stops costing you attention.
01 The three-question filter (before you even sign up)
02 The 30-minute hands-on protocol
Survivors get exactly thirty minutes, on YOUR work — never the demo's: minutes 0-10, run your three most recurring real tasks (from your workflow cards — this is why you have them); minutes 10-20, head-to-head the same tasks against your incumbent lane owner; minutes 20-30, stress it — the weird edge case, the long document, the thing your current tool fumbles. Verdict time: it must be meaningfully better, not marginally — switching costs are real, and 10% better loses to installed.
No pricing page (you're the product or the beta tester). Screenshots but no free hands-on. "Powered by GPT" as the entire feature list. Benchmark charts without task examples. Urgency marketing ("founding price ends Friday" — sound familiar? We do it too; it's a flag, not a felony — but stack it with others and pass.)
03 'Watch' is a real verdict
Most interesting tools aren't ready-for-you YET — wrong integrations, missing feature, too young. Keep a watch list with one line each: what would change the verdict ("adopt when it connects to QuickBooks"). Re-evaluate on that trigger, not on hype cycles. This converts FOMO into a filing system — and it's exactly how we run this site's coverage, which is why tracks update when tools actually change rather than when press releases drop.
04 The annual truth
Run this honestly for a year and you'll adopt two or three tools, watch a dozen, and skip fifty — and the two or three will matter more than the fifty would have. Evaluation discipline isn't resistance to new things; it's what makes you genuinely fast on the ones that deserve it.
Take the AI tool currently making noise in your feed. Run the three-question filter right now — most evaluations end here, in ninety seconds, for free.
Open the Playground →This week's challenge
Run the full protocol on one new tool this week — filter, 30 hands-on minutes against your incumbent, written verdict with the watch-trigger if it's a watch. Save the verdict. Ten of these and you'll have the rarest thing in AI: opinions you can defend.