Core LLM & training: efficiency and scaling breakthroughs
Key Questions
What scaling breakthroughs are highlighted in recent LLM training research?
GFlowRL scales distribution-matching RL to 235B MoE models, while Ring-Zero extends zero RL to 1T parameters. HiLS Attention supports 64x context extrapolation.
What is GLM 5.2 and why is it notable?
GLM 5.2 is an open-weight 744B MoE model released by Zhipu AI that outperforms many existing models in benchmarks. It features 256 experts per layer and has gained attention amid Western AI discussions.
How does on-policy distillation support LLM recovery after domain-specific fine-tuning?
On-policy distillation serves as an actionable technique for recovering model performance following domain SFT. It is presented as a practical method in current training workflows.
GFlowRL scales distribution-matching RL to 235B MoE, Ring-Zero scales zero RL to 1T parameters. HiLS Attention enables 64x context extrapolation. On-policy distillation for recovery after domain SFT is an actionable technique. New: GLM 5.2 open-weight 744B MoE with 256 experts per layer.