Novel ML Architectures
Key Questions
What is MegaTrain?
MegaTrain enables full precision training of 100B+ parameter LLMs on a single GPU. It overcomes hardware limitations for efficient large-scale model training.
What does ThinkTwice optimize in LLMs?
ThinkTwice jointly optimizes LLMs for reasoning and self-refinement. It enhances logical capabilities through iterative improvement processes.
What is Cog-DRIFT?
Cog-DRIFT enables models to learn from zero-reward examples using RLVR techniques. It advances reinforcement learning in reward-sparse environments.
What is TriAttention?
TriAttention uses trigonometric KV compression for efficient long reasoning. It reduces computational overhead in extended sequence processing.
What improvements does LightThinker++ bring?
LightThinker++ advances from reasoning compression to memory management in LLMs. It optimizes resource use for complex inference tasks.
What is Self-Execution Simulation?
Self-Execution Simulation improves coding LLMs by simulating execution during training. It boosts performance on programming benchmarks.
What is the Geometric Alignment Tax?
The Geometric Alignment Tax compares tokenization vs. continuous geometry in scientific FMs. It highlights limitations of discrete representations in geometric tasks.
What is PLUME?
PLUME is a latent reasoning-based universal multimodal embedding model. It unifies processing across vision, language, and other modalities.
MegaTrain single-GPU 100B; ThinkTwice reasoning/self-refinement; MMEmb-R1 multimodal; pruning hierarchies; In-Place TTT; Cog-DRIFT RLVR zero-reward; TriAttention KV comp; LightThinker++ memory; Self-Execution; Peking AI4Math Anderson proofs; neuro-symbolic dual memory; Chollet/Marcus symbolic; PLUME latent; test-time adaptation; geometric tax sci FMs; noisy reasoning; wetware FORCE; CoreThink etc. Hybrid efficiency challenging scaling.