Next-gen multimodal models for vision, video, and embodied AI

This cluster tracks rapid advances in multimodal and vision-language AI, especially around long, complex video understanding, controlled video generation, and 4D world and human–scene reconstruction for simulation. New benchmarks like MMOU, VET-Bench, SocialOmni, and gesture-based egocentric QA probe models’ limits in tracking entities, social cues, and extended temporal reasoning, while works like NanoVDR and redundancy-aware generation target efficiency and smaller deployable models. On the modeling side, Gaussian-splatting SLAM, physics-in-the-loop reconstruction, and omni / “transfusion” pretraining frameworks push from static perception toward interactive, embodied, and socially aware agents that can both understand and generate rich audiovisual environments. Together, these reposts highlight a shift from generic vision-language chatbots to specialized, evaluation-driven systems grounded in real-world video, documents, and physical scenes.