Evaluation integrity, real-world task performance, and agent efficiency
FDE-Gym reportedly found only 14 of 95 real customer-delivery tasks fully resolved by its strongest setup. MMEnterprise-Bench highlights evidence sufficiency and temporal or cross-source conflicts, with retrieval gains not reliably translating to answer accuracy; ANTE’s reported SWE-bench gains and other results still need replication and workflow validation. Reward validity, wrapper overfitting, costs, and synthetic-to-real transfer remain key checks.
Sources (27)
Updated Oct 21, 2026