Advances in Multi-Agent Systems and Agent Reliability
Key Questions
What are the key advances in multi-agent orchestration frameworks?
Orchestra-o1 introduces an omnimodal multi-agent orchestration framework achieving 12-18% gains. Other systems like Sakana AI's Fugu and SciOrch focus on coordinating expert LLMs and models to challenge single-model scaling.
How does VeriTrip address reliability in travel planning agents?
VeriTrip provides a verifiable benchmark for travel planning agents using noisy web data, exposing retrieval-reasoning gaps. It highlights issues in agent reliability for real-world tasks.
What improvements does StreamMA offer for agent communication?
StreamMA reduces latency and improves accuracy via streaming communication and a step-level scaling law. It enables more efficient multi-agent interactions.
What benchmarks evaluate long-horizon agent performance?
Workflow-GYM, WeaveBench, and Long-Horizon-Terminal-Bench test long-horizon GUI and terminal tasks, with top models achieving around 30% or lower success rates. These reveal ongoing reliability challenges.
How are self-evolving agents advancing?
Papers like EvoDS, MMPO, and Xiaohongshu's Evolving-RL show gains up to 98.7% over baselines on tasks like ALFWorld. They focus on avoiding capability collapse and iterative improvement.
What new tools support agent memory and skills?
MemTrain offers self-supervised memory training with 17+ point gains, while WorldMemArena benchmarks multimodal memory. Systems like A-TMA address ghost memory issues in persistent agents.
Which companies released major agent-related tools recently?
Microsoft launched MAI-Thinking-1 and Claude Science Workbench, while NVIDIA released open-source physical AI agent tools. MiniMax introduced Agent 1 for 24-hour autonomous operation.
What insights exist on agent communication protocols?
Comparisons of MCP vs A2A vs ACP provide practical references for how AI agents communicate. GRASP introduces RL for granularity-aware search in agentic RAG.
A wave of research tackles key agent bottlenecks: Orchestra-o1 introduces an omnimodal multi-agent orchestration framework with 12-18% gains. VeriTrip provides a verifiable benchmark for travel planning agents using noisy web data, exposing retrieval-reasoning gaps. StreamMA reduces latency and improves accuracy via streaming communication and a step-level scaling law; TELBench/DRIFT enables process-level error localization; MemTrain offers self-supervised memory training (17+ point gain); MMG2Skill converts web guides into self-evolving skills; a new scaling law (Effective Feedback Compute) redefines agent harness efficiency; WorldMemArena benchmarks multimodal agent memory; MapAgent achieves 95% automation in city-scale lane mapping. New papers on self-evolving agents: EvoDS achieves 28.9% improvement; Continual Experience Internalization reveals design principles to avoid capability collapse; MMPO reaches 97.1% performance at 1.75M tokens; DataCOPE improves reasoning by 32%. AdaPlanBench exposes adaptive planning weaknesses (best model 67.75%). A Stanford study finds two coding agents perform 50% worse than one alone. AXPO (Agent Explorative Policy Optimization) uses RL to fix the thinking-acting gap in multimodal agents. SePO introduces self-evolutionary system prompt optimization. Xiaohongshu's Evolving-RL achieves 98.7% improvement over GRPO on unseen ALFWorld tasks. Microsoft launches MAI-Thinking-1 (35B active, 128K context) with Scout agent and Execution Containers for agent-native Windows. NVIDIA released a major collection of open-source physical AI agent tools and skills. Bayesian-Agent uses posterior-guided skill evolution (80→95% on SOP-Bench). SkeMex enables generalizable medical agent reasoning. OmniGameArena introduces a UE5 benchmark with Improvement Dynamics Curve. SWE-Explore benchmarks coding agent repository exploration. SpatialWorld benchmarks interactive spatial reasoning (GPT-5 at 17.4% TSR). Skill-RM unifies heterogeneous reward signals. SearchSwarm achieves SOTA on BrowseComp (68.1%). Workflow-GYM benchmarks long-horizon GUI tasks (top models ~30% success). Role-Agent bootstraps agents via dual-role evolution (4% average gain). SGDR improves web agent skill learning (10% relative gain on WebArena). EEVEE enables test-time prompt learning. HiViG introduces test-time intervention for long-horizon GUI tasks. InternVideo3 agentifies foundation models. EgoBench provides interactive egocentric multimodal benchmark. WeaveBench benchmarks long-horizon computer-use agents. HarnessBridge introduces a learnable bidirectional controller. Google I/O 2026 unveiled Spark AI Agent. AutoMedBench benchmarks medical auto-research. LARA scales robot foundation models. Microsoft's FastContext (4B) provides efficient code retrieval. NVIDIA XR AI brings agents to AR glasses. LoopCoder-v2, a 7B model, achieves 64.4 on SWE-bench Verified with two loops, beating 30x larger models. SciOrch introduces learning to orchestrate expert LLMs. New: Optimal stopping strategies for multi-agent deliberation reveal a log-linear tradeoff with fatigue. SkillWeaver decomposes queries for multi-skill composition. ENPIRE enables agentic robot policy self-improvement. ARD standard for agent discovery backed by Google, Microsoft, NVIDIA. NatureBench reveals best agents only match published SOTA on 17.8% of Nature-family tasks. DeepMind explicitly states large-scale AI agent deployment is unsafe today. A principled system-level evaluation of agent memory finds no single architecture dominates. A new paper reframes agent memory as a full data-management stack. A conceptual framework defines agents across five dimensions. OPID introduces on-policy skill distillation for agentic RL. Autodata trains an agentic data scientist. Sakana AI's Fugu orchestrator beats Claude on SWE Bench Pro by coordinating other models, challenging single-model scaling paradigm. A new conceptual piece describes using self-evolving agent trajectories to train VLMs and video models via a 'director agent' pipeline, aligning with the agentic training trend. CodeChat-Eval benchmark reveals functional correctness drops 19-69% over 10 turns in multi-turn code refinement, highlighting agent reliability gaps in coding tasks. New systems: ASPIRE autonomously writes and refines control programs for robotics, building reusable skill libraries with major gains on LIBERO, Robosuite, BEHAVIOR-1K and zero-shot sim-to-real transfer. BioInsight orchestrates multi-agent biomedical knowledge discovery with interactive evidence-centered interfaces. DiscoPER achieves autonomous scientific discovery via iterative meta-reflection, recovering 8/9 patterns on iNatDisco. AutoTrainess automates LM post-training as an agent workflow, improving from 23.21 to 26.94 on CLI benchmarks and generalizing to DeepSeek-V4-Flash. ABot-M0.5 introduces dream-forcing for mobile manipulation world action models. Perceive-to-Reason decouples perception and reasoning for fine-grained visual reasoning with PRA-GRPO. Domain Arithmetic enables one-shot VLA adaptation under environmental shifts via weight arithmetic. MemSyco-Bench benchmarks sycophancy in agent memory across five tasks. XSkill introduces dual-stream continual learning from experience and skills (training-free, visually-grounded, consistent gains across 4 backbones and 5 benchmarks). New benchmarks and tools: EvoPolicyGym evaluates autonomous policy evolution; DiscoBench tests clarification-aware deep search; PACE provides a proxy for agentic capability evaluation (<4% MAE); AgenticDataBench covers 15 domains for data agents; AgenticSTS tests bounded-memory long-horizon agents; SkillCoach offers self-evolving rubrics for skill-use evaluation. New: Sakana AI's Sheaf-ADMM multi-agent coordination paper at ICML 2026. DuoMem achieves 77.9% on ALFWorld with 4B model via dual-space distillation for on-device memory agents, with 3x speedup and tiny memory footprint. AdaJEPA introduces adaptive world models that continuously learn from closed-loop interaction, relevant for agent planning. Anthropic launched Claude Science Workbench, an agentic workflow for scientific research with ontology mapping and provenance tracking. MiniMax launched Agent 1, achieving 24-hour autonomous operation with visual understanding and multi-platform coordination. Unified Decision Language Models reformulate offline multi-agent RL as dialogue-style sequence prediction using SFT+GRPO, outperforming baselines with zero-shot generalization. These developments push toward more reliable, autonomous agent systems. New practical insight: Ghost memory in long-running agents—old facts persisting and misleading—is addressed by A-TMA's state-aware overlay with evidence packets, a practical fix for persistent assistants. New: EVA-Client provides a unified open-source framework for embodied policy data collection, inference, and deployment on real robots. UI-MOPD tackles catastrophic forgetting in multi-platform GUI agents via on-policy distillation with platform-specific teachers, achieving 38.2% on OSWorld and 12.0% on MobileWorld. A new constrained decoding method from ICML2026 guarantees successful tool calls with bounded tokens, improving agent reliability. New papers: Light-Omni achieves 12.1x speedup and 2.6x memory efficiency for agentic video understanding via reflexive design with dual contextual states. SkillOpt-Lite formalizes agent skill optimization as minimal ZO pipeline, achieving +25.4 on LiveMath and enabling nano models to outperform larger ones. TREK uses distillation to expand exploration support for GRPO, showing gains on AIME and ALFWorld. A new verification framework extracts continuous scores from LLM judges and uses probabilistic tournaments for efficient ranking, with strong results on agent benchmarks. New: LaMem-VLA introduces dual latent memory (short-term and long-term) for VLA models in robotic manipulation, operating entirely in the VLA's native embedding space, addressing Markovian limitations in long-horizon tasks. Recent additions: Automating Embodied Agent Architectures (AgentCanvas, KDLoop) tackles automated design of embodied agent architectures with thorough evaluation across VLN, EQA, and manipulation. SAO (Single-Rollout Asynchronous Optimization) improves GRPO stability for long-horizon agentic tasks with single-rollout sampling and double-side clipping, deployed in GLM-5.2. AgentLens provides production-assessed trajectory reviews for coding agent evaluation. Meta Muse Spark 1.1 introduces explicit multiagent orchestration with main agent + subagents, context compaction, and computer use. A new ZendoWorld benchmark reveals VLM agents fail at active visual concept induction—proposing near-uninformative experiments despite good labeling accuracy, exposing a critical gap in hypothesis-driven exploration. A new cognitive-structured multimodal agent (8B) beats 32B baselines by 8.2% while halving inference time via episodic memory and visual token compression, challenging monolithic scaling. UniClawBench provides a universal benchmark for proactive agents in real-world Docker environments, testing five foundational capabilities with closed-loop evaluation. François Chollet notes mind-blowing speed of agentic coding progress. New: Long-Horizon-Terminal-Bench reveals best model only 15.2% at 0.95 threshold, mean 4.3%, challenging agent reliability for long-horizon tasks. New: AgentCompass provides a unified evaluation infrastructure for agent capabilities with decoupled Benchmark/Harness/Environment design and trajectory analysis for reward-hacking detection. New: Self-Improvements in Modern Agentic Systems survey provides a structured overview of self-improving agents. New: Microsoft and colleagues work on debugging agent trajectories at scale. New: PalmClaw is a native on-device agent framework for mobile phones, achieving 11.5% better task success and 94.9% faster completion. New: SearchOS-V1 introduces explicit state management (Frontier Task, Evidence Graph, Failure Memory) for robust multi-agent information seeking, with pipeline-parallel scheduling. Recent reading: GRASP introduces RL for granularity-aware search in agentic RAG, learned skimming/scanning behavior. Rethinking harness evolution: reported gains may be artifacts of search budget, evolved harnesses generalize poorly. MCP vs A2A vs ACP protocol comparison provides practical reference for agent communication. These developments push toward more reliable, autonomous agent systems.