Frontier AI Insights

Stat physics phase trans to LLMs/Hopfield; spontaneous symmetry breaking (Bronstein); non-eq DMFT/double descent; RMT/grokking; EBMs (LeCun); Rigorous Theory of LLMs (Brock); control theory; Sara Hooker post-scaling; unifying physics/neuroscience/AI. New: Hamilton-Jacobi Theory of Deep Learning (exact PDE correspondence, quantitative generalization/robustness/influence functions); Geometric Latent Reasoning (GLR) induces shorter generations; 'Why Larger LLMs Learn Rare and Complex Tasks' (reduced interference); Local Perturbation Theory for Multi-Domain RL (second-order effects); Influence-Guided Symbolic Regression (LLM+MCTS); Shay Moran talk on geometry of ranking; Latent Prediction beats token-level training (exponential sample efficiency); Omar Sar follow-up (evolver plateaus, solver inverted-U); Jing Huang paper questioning larger models; MUX method (latent continuous reasoning); Meta-Agent Challenge (frontier models struggle to self-improve); Noam Razin talk on proxy reward functions; AI solves Grothendieck constant; Meta-Cognitive Memory Policy Optimization (97.1% at 1.75M tokens); Shadow Price of Reasoning (CLEAR 3x accuracy). Also MARS automates AI research; STRIDE training data attribution via activation space. New: Self-Revising Science Agents via Category Theory (copresheaves to distinguish routine search from true discovery); @blader announces breakthrough in self-evolving AI scientists moving from search to principled discovery. SePO introduces self-referential evolutionary system prompt optimization, showing consistent gains and pre-training generalization. SIA unifies harness and weight updates in a single self-improving loop (56.6% LawBench gain, 502% denoising). OpenSkill bootstraps skills and verification from scratch for open-world self-evolution without target-task supervision. New: HarnessForge formalizes harness-policy co-evolution for LLM agents (12% gain). KnowSelf introduces self-knowing agents that detect knowledge gaps and strategically acquire skills. New: Experience Makes Skillful (SkeMex) for medical agent self-evolution with structured skill memory; SEE elicits latent judge calibration with minimal data (160 examples); PBSD addresses long-horizon credit assignment via Bayesian self-distillation. Also Math Theory of Deep Representation Learning (unifying memory and world models). Cosine Misleads paper challenges latent visual reasoning assumptions in VLMs. New agent self-improvement papers: Role-Agent (dual-role evolution), RHO (retrospective harness optimization, 59%→78% on SWE-Bench Pro), SearchSwarm (delegation intelligence, SOTA on BrowseComp), EEVEE (test-time prompt learning for heterogeneous streams). Also UC San Diego Professor Daniel Kane awarded Gödel Prize for robust high-dimensional statistics. New: Deficient executive control in transformer attention paper identifies fundamental failure mode. EvoTrainer co-evolves LLM policies and training harnesses for autonomous agentic RL. New: Daniel Barzilai (Weizmann) talk on model collapse and SGD lower bounds—model collapse positive under regularity, negative without structure; SGD lower bounds via direction-finding difficulty. Edward Lockhart (DeepMind) talk on formal mathematics as solution to reward hacking. New: HarnessBridge introduces learnable bidirectional controller for LLM agents, extending harness-policy co-evolution. New: OPD geometry paper reveals sparse, FFN-concentrated updates with spectral signatures. New: Luca Baggi talk on mechanistic interpretability synthesizes recent work on circuit tracing and sparse features. New: ICML position paper on distinguishing genuine deception from role-play in AI (Mitchell et al.) forces more rigorous experimental design in alignment research.