Open Models Commoditize and Scale
Key Questions
What notable open models were recently highlighted?
DeepSeek V4, a 1.6T MoE model, and Kimi K3, which tops web engineering benchmarks, are key examples. Kimi K3 weights are scheduled to arrive July 27 and can serve as a teacher model for distillation.
How can Kimi K3 be used in model training?
It supports training smaller dense models using logits from the teacher model, as noted by @EMostaque. This approach enables efficient distillation while maintaining high performance.
What does the collapse of pricing moats mean for open models?
Gary Marcus notes that open models like Kimi K3 are deflationary for big tech by eroding pricing advantages. Natolambert argues they are accelerationist overall.
What scaling developments are expected in open AI models?
A scaling arms race continues with 10T parameter models on the horizon. This commoditization trend is reshaping competitive dynamics in the field.
What legal actions involve major AI companies?
Apple is suing OpenAI over alleged trade secrets, reflecting intensifying competition amid open model advancements.
DeepSeek V4 1.6T MoE, Kimi K3 tops web engineering benchmark (weights arriving July 27, usable as teacher model for distillation), pricing moat collapses (Gary Marcus). Natolambert argues open models are accelerationist and deflationary for big tech. Scaling arms race: 10T parameter models coming. Apple sues OpenAI over trade secrets.