Unified Algorithms for Foundation-Model Training

New work proposes unifying pretraining, supervised fine-tuning, and reinforcement learning under a likelihood-based training framework, emphasizing data composition and scheduling. Reported experiments at roughly 124M and 760M parameters claim improved compute tradeoffs and generalization, but evidence at this scale does not establish performance on modern foundation models.