Evaluation integrity and agent efficiency advances
Evaluation is shifting from static leaderboards toward runtime-controller tests, reusable-skill analysis, independent held-out workflow tasks, lifecycle-aware judges, harness-aware measurement, and business-relevant outcomes. LoopArena and sandbox failures show that controller quality and evaluation infrastructure can dominate measured performance, while contamination, evaluator leakage, reward hacking, reproducibility, and the distinction between reasoning gains and extra search remain unresolved.
Sources (3)
Updated Sep 5, 2026