Agent Evaluation: Why It's Harder Than Model Eval

I'm building AgentEval Forge, an OSS evaluation lab for agents. The pitch sounded simple: scenario packs, adversarial cases, trajectory scoring, regression tracking, cost and latency analysis. A more structured version of eval work I'd done before. That was the first mistake. I'd spent plenty of time on model-eval and workload-eval systems, where you compare models on real tasks with rubrics, watch for speed versus cost, and know how fast a pretty score turns misleading when your harness is weak. I assumed agent evaluation was just an extension of that world. It isn't.

With a clean model eval, the center of gravity stays stable: input, output, score. Exact match, rubric, LLM judge, benchmark harness. Figure out if the model produced a good answer for this task. That's already hard—I've seen brittle scoring and false confidence look scientific. But the shape of the problem is still "did the answer land."

Agents change the shape. The thing being evaluated is no longer an output; it's a workflow. Tool choices, retries, intermediate state, cost, recovery behavior, policy adherence. The moment a system does more than respond once, the final answer stops being the whole story. That sounds obvious out loud, yet a lot of teams still score their agents the way they scored their models.

The trap I kept running into while building AgentEval Forge is that a decent final answer can hide a messy path. An agent can pick the wrong tool first and recover by luck. Loop more than it should. Burn five times the tokens a cleaner route would need. Take a risky step a human reviewer would never approve, then back into something that looks fine at the end. If you only score the final answer, that run gets marked as a success. That's the trap.

The path matters. That's the sentence I keep coming back to: the path matters as much as the endpoint,