New Benchmarks for Agents, Editing and Memory
Key Questions
What benchmarks are introduced for evaluating AI agents?
TerminalWorld and SpaceDG provide new rigorous benchmarks for real-world terminal tasks and spatial intelligence under visual degradation. Evaluations use Terminal-Bench's standard Harbor harness for consistent agent testing.
How does DexJoCo support research in dexterous manipulation?
DexJoCo offers a benchmark and toolkit for task-oriented dexterous manipulation built on MuJoCo. It includes video demonstrations and evaluation tools to advance robotics research.
What is the scale of the MINTEval benchmark?
MINTEval introduces a large-scale memory evaluation benchmark with 1.8M tokens. It focuses on advancing assessment of long-context memory capabilities in models.
How does SpaceDG test spatial intelligence?
SpaceDG benchmarks spatial reasoning under conditions of visual degradation. It provides datasets and tasks to measure robustness in challenging visual environments.
What evaluation methods are used for motion editing in CVPR work?
MotionEdit advances evaluation protocols for motion editing tasks. These methods enable more precise assessment of model performance in dynamic editing scenarios.
TerminalWorld and SpaceDG introduce rigorous agent terminal and spatial robustness benchmarks; MINTEval (1.8M tokens) and MotionEdit CVPR advance memory and motion editing evaluation.