Self-evolution & memory / runtime learning
Key Questions
What is SEED in the context of agentic reinforcement learning?
SEED is a self-evolving on-policy distillation method that converts on-policy trajectories into hindsight skills and distills them back into the policy to address supervision gaps in sparse-reward tasks.
How does MemCon improve memory management for LLM agents?
MemCon uses a lightweight bandit to decide when to retrieve, plan-inject, re-retrieve, or consolidate memory, achieving up to 15.2 point gains with 5-20% token reduction and no extra LLM calls.
What are the main findings from the Self-Improvements in Modern Agentic Systems survey?
The survey formalizes self-improvement as a self-induced update operator on model parameters or scaffold components and provides a system-level framework for understanding agent improvements.
What does Liu's three-layer taxonomy for self-improving agents cover?
It categorizes improvements into artifact, harness, and model layers, noting that most gains come from harness-level changes rather than weight updates.
How does ATHENA-R1 perform on medical reasoning tasks?
ATHENA-R1 is an RL-trained agent that reaches 94.7% accuracy on FDA questions and 82.9% on patient-specific decisions, showing tool-use skills matter more than model scale.
MOSS, RubricEM, SkillOpt, SIA (502% gain), NVIDIA Polar, AHE (69.7→77%). Ornith-1.0 learns own RL scaffold. Eevee (+37-48%). Retrospective Harness Optimization 59%→78% SWE-Bench Pro. APPO, EvoTrainer, Arbor. DeepIndex agentic memory (118K tokens/query). ByteDance post-deployment scaling law — agents double learning speed every 3 months. ContReAct, HiPER, DuoMem. Self-evolving multi-agent via decentralized memory. Understanding Reasoning Collapse in LLM Agent RL — MI-based diagnostic and reward-variance-aware filtering for multi-turn agent training. SkillOpt-Lite — minimal ZO optimization pipeline for skill evolution; nano model outperforms larger full SkillOpt; integrated into VSCode Copilot; challenges complex pipeline assumptions. AReaL 2.0 — three-pillar architecture (ATDP, data proxy, control plane) for turning production failures into training signals; practical infrastructure for self-evolving agents. IGRPO — information gain-based rollout policy optimization for tree-structured search in LLM agents; principled rollout budget allocation; consistent gains on 7 QA benchmarks. TurnOPD — turn-level on-policy distillation for long-horizon agent training; reduces wasteful full-horizon rollouts and unbalanced KL loss; improves accuracy-time frontier. LaMem-VLA — dual latent memory in VLA models for robotic manipulation; embeds short/long-term memory directly into latent space, addressing Markovian assumption bottleneck; enables fluid interleaving of history with reasoning. New: Liu's three-layer taxonomy for self-improving agents (artifact, harness, model) — cuts through hype, most gains from harness-level improvements, not weight updates. Practical framework for builders. New: In-Context RL via Communicative... — frames in-context RL as emergent communication between world model and control agent using causal influence loss; zero-shot adaptation in sparse-reward environments; practical for sample efficiency. New: TRACE — capability-targeted agentic training system; contrastive capability analysis + targeted synthetic envs + MoE composition; 27B model beats GPT-5.2-Codex on SWE-bench. New: ATHENA-R1 — RL-trained agent for medical treatment reasoning; two-stage (synthetic data + RL); tool-use skill matters more than model scale; 94.7% accuracy on FDA questions, 82.9% on patient-specific decisions. New: MemCon — learned adaptive memory management for LLM agents; lightweight bandit decides when to retrieve, plan-inject, re-retrieve, or consolidate; zero extra LLM calls, up to 15.2 point gains with 5-20% token reduction; backend-agnostic. New: Self-Improvements in Modern Agentic Systems survey — formalizes self-improvement as self-induced update operator on model parameters or scaffold components; system-level framework. New: Identity layer for embodied agents that keep learning — proposes 'governable individual' with signed lifecycle transitions to address governance gap when agents rewrite themselves. New: SEED — Self-Evolving On-Policy Distillation for Agentic RL; converts on-policy trajectories into hindsight skills and distills back into policy; addresses supervision gap in sparse-reward tasks. New: Podcast with Qizheng Zhang on recursive self-improvement — harness-level and weight-level improvements are complementary, not competing; practical perspective for builders.