World Models Frontier: JEPA, Action Models, Spatial & VLMs
Key Questions
What is OpenWorldLib?
OpenWorldLib is a unified codebase and definitions for world models. It standardizes research in JEPA, action models, and spatial VLMs.
What is LeCun's LeJEPA and SigReg?
LeJEPA (Joint-Embedding Predictive Architecture) uses SigReg for physical planning. It advances predictive world models beyond pixels or 3D.
Do World Action Models outperform VLAs?
World Action Models generalize better than Vision-Language-Action models (VLAs) in robustness studies. They handle dynamic environments effectively.
What are the Three Levels of TTT?
Three Levels of TTT include Test-Time Training, Meta Training, and World Modeling. They enhance agent adaptation in spatial tasks.
What is Token Warping in MLLMs?
Token Warping enables MLLMs to view from nearby viewpoints. It improves spatial understanding in multimodal models.
What is Stanford EgoNav?
Stanford EgoNav uses a camera for campus navigation over 5 hours. It demonstrates real-world spatial world modeling.
What biases affect VLMs?
VLMs ignore visual details for semantic anchors and exhibit bias. Latent taxonomy and GaussianGPT (3D generation) address these.
What datasets support world modeling?
WildWorld is a large-scale dataset for dynamic modeling with actions and states. It targets general intelligence via explicit state tracking.
OpenWorldLib unified codebase/defs; LeCun LeJEPA SigReg; World Action Models > VLAs; Three Levels TTT; Token Warping MLLMs; Stanford EgoNav; GaussianGPT/SpatialLM; latent taxonomy; VLM semantic bias.