Frontier Multimodal Models and Physical AI

Major model releases: Meituan LongCat-2.0 (1.6T MoE, 48B active, 1M context, SWE-bench Pro 59.5, open weights MIT, trained on domestic ASICs), Meta Muse Spark 1.1 (closed, hosted, multiagent orchestration, 1M context, zero-shot tool generalization, safety evaluations, beats GPT-5.6 on SciCode, strong on theoretical CS induction reasoning), NVIDIA Cosmos 3 (open physical AI foundation model), Google Gemma 4 12B (encoder-free multimodal, native audio, runs on 16GB VRAM, Apache 2.0), Qwen 3.7 Plus (multimodal agent), Echo-Infinity achieves 24-hour real-time infinite video generation. NVIDIA Nemotron 3 Ultra (550B MoE hybrid Mamba-Transformer). MiniMax M3 (open-weight, 1M context, native multimodal), Ideogram 4, TripoSplat. Cognition's SWE-1.7 (RL training details: entropy collapse via top-p sampling replay, Muon optimizer, multi-continent rollout). New embodied AI papers: Qwen-VLA unifying vision, language, and action for robotics; 'Where to Look' testing active exploration. ReasonMatch-Bench reveals a massive gap (human 84.0 vs best model 37.2 F1), with DCRL offering a reinforcement learning approach. VideoKR introduces 315K expert-domain examples with CoT for video understanding. LoomVideo is a compact 5B model unifying video generation and editing. The World-Language-Action (WLA) model achieves 92.94% on RoboTwin2.0 with 40ms inference. Discrete-WAM unifies discrete vision and action tokens for world-policy learning. Future-L1 achieves +24.4 points on FutureBench. Liquid models emerge as a post-Transformer architecture candidate. NVIDIA alpamayo-R1 (first open-weight autopilot model) and 4D-RGPT for native 4D understanding. TV2TV at CVPR, a unified text-video generation model. VLA-JEPA released in LeRobot. Flash-WAM achieves 23x speedup for world action models. VLMs as teachers improve video reasoning. AdaCodec reduces token budget for video MLLMs by 7x. Microsoft launches MAI-Thinking-1 (35B active, 128K context) reasoning model. Stanford's 2026 AI Index confirms frontier models match/exceed human experts on PhD-level science exams. NVIDIA's Locate Anything introduces Parallel Box Decoding. Google's Gemma 4 12B offers efficient on-device multimodal. Qwen-VLA unifies vision, language, and action. AnchorWorld tackles interactive egocentric world simulation. Stream3D-VLM enables online 3D understanding. Liquid AI introduced a method to build multimodal models without multimodal training data. Latent Spatial Memory for video world models achieves 10.57x faster generation. AHA-WAM decouples world prediction from action execution. CADGenBench benchmarks engineering-grade 3D part generation. Microsoft's Mirage uses latent spatial memory. MemDreamer achieves SOTA on long video understanding. ABot-Earth 0.5 generates realistic 3D environments from satellite imagery. A new ICML 2026 paper on efficient generative modeling beyond memoryless diffusion. LLM-Guided Neural Architecture Search for robust co-design. Keye-VL-2.0 is a 256K context MoE multimodal model. Embodied-R1.5 targets physical intelligence. A new world model uses 2D stick-figure skeletons for cross-embodiment generalization. Wan-Streamer achieves ~200ms model latency for end-to-end real-time multimodal interactive models. Causal-rCM extends diffusion distillation to autoregressive video generation with 10x faster convergence and SOTA VBench-T2V score (84.63). iLLaDA scales a bidirectional diffusion LM to 8B/12T tokens. Google I/O 2026 unveiled Gemini Omni. Kimi K2.7 Code (1T MoE) for agentic workflows. Zamba2-VL hybrid Mamba2-Transformer VLMs cut TTFT by ~10x. LARA scales robot foundation models. MVEB provides a massive video embedding benchmark. A new paper on sub-quadratic vision transformers addresses O(n²) bottleneck. GLM-5.2 offers 1M-token context. Variable-Width Transformers propose rethinking width as a scaling axis. SimSMoE targets efficient training of sparse MoE models. Kairos introduces a native world model stack for physical AI. Noam Shazeer leaves Google for OpenAI after Google paid $2.7B to rehire him. Human video to 4D robot hand-object trajectories addresses data bottleneck. A systematic evaluation of multimodal CoT reveals it's not a free lunch. Gary Marcus and Eric Topol flag that frontier models fail at multimodal medical reasoning in a Nature Medicine stress test. Alibaba released Qwen-Image-Agent for agentic image generation, planning, reasoning, and searching. New: Seed2.0 from ByteDance/Seed claims world-leading reasoning, visual understanding, and search for real-world complexity. ABot-M0.5 introduces dream-forcing for mobile manipulation world action models. Perceive-to-Reason decouples perception and reasoning for fine-grained visual reasoning with PRA-GRPO, achieving strong gains on V-Star and HR-Bench. Domain Arithmetic enables one-shot VLA adaptation under environmental shifts via weight arithmetic. Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning addresses train-inference mismatch in latent reasoning, achieving +10.83 on BLINK. SoftMoR enables deeper recursive vision transformers via soft mixture of intermediate representations (82.48% top-1 on ImageNet with 22.7M params, beating DeiT-B). Task-Agnostic pretraining for VLAs decouples motor priors from language grounding, matching 1M expert trajectories with far less data and achieving 25% success under perturbations vs 0%. New papers: WorldDirector builds controllable world simulators with persistent dynamic memory; AnyGroundBench reveals VLMs fail in specialized video grounding domains; FlashMorph optimizes hybrid attention layer selection. A large-scale NAS study (HARMONY) on Frontier finds heterogeneous mixing of Transformer, MoE, and Mamba-2 outperforms homogeneous, with best config at 2.38B params, 1.0874 perplexity, 4320 tok/s. A principled construction for resolution-agnostic neural operators enables CNNs and transformers to generalize across grid resolutions. AdaJEPA introduces adaptive world models that continuously learn from closed-loop interaction. Embodied.cpp provides a portable C++ inference runtime for embodied AI models on heterogeneous robots, addressing deployment fragmentation. DataComp-VLM benchmarks VLM data curation, finding data mixing (especially instruction-heavy) beats filtering with +5.4pp over FineVision. New: PixWorld unifies 3D scene generation and reconstruction in pixel space, avoiding latent space information loss with geometry perception loss. New papers: RynnWorld-Teleop (action-conditioned world model for digital teleoperation, 40+ FPS, zero-shot Sim2Real), RynnWorld-4D (4D embodied world models, 254M-frame dataset, SOTA bimanual), AlayaWorld (long-horizon playable video world generation, open-source), Nemotron-Labs-Diffusion (tri-mode LM unifying AR, diffusion, self-speculation, 6x throughput), HiLS Attention (hierarchical sparse attention for infinite context, 64x extrapolation), MuseBench (intent-level audiovisual arts understanding, best MLLM 48% vs human 87%). New: LingBot-Video is a DiT-based MoE video pretraining model specifically for embodied intelligence, using robot-oriented data and multi-dimensional reward for physical realism, open-source. RoboDojo provides a unified sim-and-real benchmark for generalist robot manipulation policies with 42 sim tasks and 18 real tasks across 5 capability dimensions, with 30 policies already evaluated. LaMem-VLA introduces dual latent memory for VLA models in robotic manipulation, addressing long-horizon tasks. Recent additions: Co-LMLM uses continuous vector queries for knowledge externalization, beating models trained on 40x more data and matching gpt-4o-mini on factuality, with editable KB and unlearning support. OmniTacTune achieves 85-100% success in tactile adaptation with 40-80 min fine-tuning via policy-agnostic real-world RL. Wake up for Touch adds tactile reasoning to MLLMs via mask-isolated alignment without catastrophic forgetting. Video-Oasis reveals 55% of benchmark samples are solvable without visual or temporal context, challenging multimodal evaluation rigor. Jeff Dean's group achieves SOTA on fine-grained subtask annotation for VLA models with cross-embodiment generalization, achieving high F1 on REASSEMBLE and blade insertion. New: Thinking Machines (Mira Murati) released Inkling, a 975B MoE open-weight model (Apache 2.0) with strong agentic tool use (74.1% MCP Atlas) and controllable thinking effort, trailing top Chinese models on coding but leading on voice understanding and offering censorship resistance. New: SPEAR is a photorealistic simulator for embodied AI research with 14K programmable UE functions. New: Self in Space benchmark evaluates self-awareness and spatial cognition in UAV embodied intelligence, revealing imbalance in MLLMs. New: RoboTTT: Context Scaling for Robot Policies applies test-time training to improve policy robustness. New: VideoChat3 is a fully open 4B video MLLM with I3D-ViT and adaptive frame resolution for efficient video understanding. New: Concurrent Image Understanding and Generation paper presents self-correcting coupled Markov jump processes for joint multimodal understanding and generation, releasing three large-scale datasets. New: Kimi K3 tops a comprehensive web engineering benchmark ahead of all proprietary models including Fable. Recent reading: GPT-5.6 used a prompt to close a 30-year gap in convex optimization (requires 10-page expert prompt, not peer-reviewed). These developments push the frontier of multimodal understanding and physical AI.