LLM reinforcement learning exposes distinct sources of training-inference mismatch
Research separates engine-numerics error from stale sampler weights as causes of off-policy data in LLM RL, arguing that the two need different corrections. On-policy distillation finds some rollout mismatch tolerable, but model-specific limits and prefix shifts complicate correction; small-model results need scale validation. Native NVFP4 training claims for Inkling 2 Small suggest possible end-to-end low-precision efficiency gains, but quality trade-offs and technical evidence remain to be assessed; FOAM and SimpleHC also await broader replication.
Sources (6)
Updated Nov 16, 2026