Paradigm shift in AI: Sutton's Oak Lab challenges scaling laws
Key Questions
What is Richard Sutton's main critique of current LLM scaling?
Sutton argues that pretraining on human text embeds human knowledge and contradicts the Bitter Lesson, making current scaling fundamentally broken.
What algorithm does Oak Lab introduce?
Oak Lab's NetworkIDBD algorithm learns per-parameter step sizes to avoid noise absorption and enable low-power continual learning.
Why should investors watch Oak Lab?
Oak Lab signals a potential paradigm shift that could reshape infrastructure investments and long-term AI development bets.
Turing Award winner Richard Sutton launches Oak Lab, arguing current LLM scaling is fundamentally broken because pretraining on human text embeds human knowledge, contradicting his Bitter Lesson. Oak Lab's NetworkIDBD algorithm learns per-parameter step sizes, avoiding noise absorption. This challenges the entire scaling paradigm and points toward low-power continual learning. For investors, a must-watch signal about potential paradigm shift that could reshape infrastructure and investment bets.