Evaluation methodology and agent efficiency advances
Key Questions
What is Recursive Harness Self-Improvement (RHI) and how does it improve agent performance?
RHI boosts low-reasoning agents while reducing costs by 60%. It focuses on self-improvement techniques within evaluation harnesses to enhance efficiency.
How does On-Policy Delta Distillation support reasoning transfer in agents?
On-Policy Delta Distillation enables efficient reasoning transfer between models. It helps address issues like model staleness and drift in production settings.
What does the RESOURCE2SKILL paper contribute to agent development?
RESOURCE2SKILL distills executable agent skills from human-created multimodal resources. This approach supports more compact and effective skill acquisition for agents.
What insights does Anthropic's J-space paper provide for model evaluation?
The J-space paper adds interpretability for reasoning load-bearing components. It improves the trustworthiness of evaluations by clarifying how reasoning occurs in LLMs.
What is covered in the survey of on-policy distillation failure modes?
The survey curated by @BhavinJawade lists common pitfalls and fixes for on-policy distillation. It adds practical knowledge for improving production reliability and avoiding training issues.
Recursive Harness Self-Improvement (RHI) boosts low-reasoning agents at 60% lower cost. On-Policy Delta Distillation enables efficient reasoning transfer. RAGU shows compact domain-adapted models outperform larger ones for KG construction. RESOURCE2SKILL distills agent skills from multimodal resources. Anthropic's J-space paper adds interpretability for reasoning load-bearing, improving evaluation trustworthiness. New paper 'Understanding Reasoning from Pretraining to Post-Training' studies joint scaling law and compute allocation insights, informing training cost debates. A survey of on-policy distillation failure modes (curated by @BhavinJawade) lists pitfalls and fixes, adding to production reliability knowledge. These address model staleness, drift, and production evaluation pain points.