Targeted benchmarks & diversified evaluation
Key Questions
What new benchmarks focus on embodied and spatial intelligence?
ESI-Bench evaluates embodied spatial intelligence and active exploration, while SpaceDG benchmarks spatial intelligence under visual degradation.
How do GUI and smartphone agent benchmarks work?
OmniGUI benchmarks GUI agents in omni-modal smartphone environments, and CutVerse provides a compositional GUI benchmark for media post-production.
What evaluations target video and audio generation quality?
Artifact-Bench assesses video artifacts, MSAVBench evaluates multi-shot audio-video generation, and WorldReasonBench tests reasoning capabilities.
Why is there skepticism toward LLM leaderboards?
Leaderboards often fail to capture real-world performance nuances, prompting calls for more diversified and reliable evaluation methods.
What clinical or multimodal datasets are introduced?
Gastric-X offers a multimodal clinical dataset, and CHI-Bench evaluates agents on long-horizon clinical workflows.
How does THUD expose issues in multimodal LLMs?
THUD reveals audio shortcuts that multimodal LLMs exploit, highlighting limitations in true multimodal reasoning.
What is FastGaze and OccuBench used for?
They provide targeted benchmarks for gaze estimation and occlusion handling in vision tasks.
How does Process Rewards reliability factor into evaluations?
It examines the consistency and trustworthiness of process-based reward signals in agent and model assessments.
FastGaze, OccuBench, WorldReasonBench; LLM leaderboards skepticism. New: ESI-Bench embodied spatial/active exploration, OmniGUI smartphone GUI, Artifact-Bench video artifacts, MSAVBench multi-shot AV, Process Rewards reliability, Gastric-X multimodal clinical dataset, THUD audio shortcuts in MLLMs, SpaceDG (spatial under degradation), π-Bench (proactive long-horizon). New: CVPR 2026 must-see paper list with code/demos.