Most AI evals measure the wrong thing
Benchmarks tell you how a model does on someone else's problem. Your users only care about theirs.
Last month a team I work with swapped one model for a cheaper one. The benchmark scores were nearly identical. Within a week, support tickets about wrong answers had doubled.
Nothing was wrong with the benchmark. It just wasn't measuring anything our users were doing.
Start from the job, not the model
The useful question is not how smart is this model? but does this feature still do the job our users hired it for? Those are different tests, and only one of them shows up in your retention chart.
If you can't write down twenty examples of a good answer, you're not ready to evaluate a model. You're ready to talk to users.
Write tasks like a product manager
A good eval task reads like a user story with an answer key. Here's one from our suite:
task: refund-eligibility
input: "I bought the annual plan 40 days ago, can I get my money back?"
expect:
- mentions: "30-day"
- tone: polite, not apologetic
- must_not: promise a refundThree things that actually helped
Twenty honest examples. More than that and nobody reads the failures.
A diff view. Seeing two prompt versions side by side beats any single score.
A named owner. Evals rot the moment nobody is accountable for them.
What to do on Monday
Pick the one feature that would embarrass you most if it broke, and write ten tasks for it before lunch.
That's it. That's the whole methodology. The rest is tooling.
Placeholder article: this text shows how long-form layouts look. Replace it from the Newsroom.