Research explores new ways to structure model training

A CMU-associated paper argues pretraining, supervised fine-tuning, and RL can be viewed as one likelihood-based process, making data mixtures and schedules central design choices; experiments are small and no schedule works reliably across tasks. A separate small-model proof of concept claims self-generated program sequences can support transferable prediction across modalities and later real-data learning, but does not show synthetic-only training scales or replaces conventional pretraining. Training–inference mismatch corrections and Astra’s reported weight-editing gains also need broader, independent validation.