LLM Commoditization: Open models, price war, new architectures
Key Questions
What evidence shows the commoditization of LLMs?
OpenAI reported a $38.5B loss amid open models like Zai_org trained on Huawei Ascend at 90% lower cost and open-source releases with only a 3-month lag. Amazon's Trainium chips and local 30B MoE models achieving 40 tok/sec on consumer hardware further intensify competition with Nvidia and closed models.
How do new MoE architectures and routing optimizations impact serving efficiency?
ProbMoE and StanfordHAI scaling laws improve efficiency, while ELDR routing for PD-disaggregated MoE serving reduces TPOT by 5.9-13.9%. Nemotron 3 Ultra and open-weight MoE models demonstrate competitive performance at lower costs.
What timeline is predicted for open models to match frontier capabilities?
@emollick warns of 4-8 months before Mythos-class open models emerge. Claude Opus 4.8 currently leads agentic benchmarks, with Microsoft MAI 5B reaching 51% on SWE Bench Pro.
OpenAI $38.5B loss. Zai_org trains on Huawei Ascend at 90% lower cost, open source, 3-month lag. Amazon selling Trainium chips challenges Nvidia. @rasbt shows local open-weight MoE models (30B) matching GPT 5.5 speed (40 tok/sec) on consumer hardware; Claude Code vs Codex 2x token efficiency gap. @emollick warns 4-8 month before Mythos-class open models. Claude Opus 4.8 tops agentic benchmarks. Microsoft MAI 5B 51% SWE Bench Pro. Nemotron 3 Ultra open MoE hybrid. Price war intensifies. ProbMoE improves routing. StanfordHAI scaling laws efficiency hack. New ELDR routing optimization for PD-disaggregated MoE serving reduces TPOT by 5.9-13.9%. New HOLA architecture pairs compressive recurrent state with small exact memory for long-range recall in linear attention models, achieving strong perplexity and needle recall at 16x training length.