Agent eval surge: Stanford multi-agent/ARC-AGI-3/ToolProbe/MCP-AgentBench/AMA-Bench/CaP-X/AEC/VideoZeroBench/Exgentic/Agent Evals/RAG evals/Apollo/Agentic-MME/AgentHazard/AgentSocialBench/OpenTelemetry/adversarial QA/ClawArena/Agent Harness
Key Questions
What is AMA-Bench focused on?
AMA-Bench evaluates long-horizon memory capabilities for agentic applications.
What does AgentHazard benchmark?
AgentHazard assesses harmful behavior in computer-use agents, emphasizing safety risks.
How does OpenTelemetry aid agent evals?
OpenTelemetry provides distributed tracing for agentic workflows, enabling production monitoring and adversarial QA in DevOps.
What is Agentic-MME?
Agentic-MME evaluates what agentic capabilities add to multimodal intelligence.
What does ClawArena test?
ClawArena benchmarks AI agents in evolving information environments.
What trend does MIT report on task lengths?
MIT notes LLMs doubling task lengths from 3.8 months, with over 3k tasks analyzed.
What is the Agent Harness survey about?
The survey covers taxonomy, challenges, and sandboxing for LLM agent harnesses.
What does AgentSocialBench evaluate?
AgentSocialBench assesses privacy risks in human-centered agentic social networks.
Stanford: single agents > multi-hop multi-agent equal tokens (debunks hype); AMA long-horizon; Exgentic coord/safety; Agent Evals OpenTelemetry/adversarial QA; ARC-AGI-3/Galtea/EVA/Omni/CaP-X/SlopCode/FinMCP/ToolProbe/MiroEval/HippoCamp/AEC/VideoZero/ClawArena; Agentic-MME/AgentSocialBench/AgentHazard; RAG DeepEval; Agent Harness taxonomy; MIT task lengths double; Apollo self-preserve. Ties swarms/robotics.