mlsanityData to 7 Oct 2026

MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation

arXiv 2602.09878 ↗First posted 10 Feb 2026cs.CVcited by 13

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World-model-based imagine-then-act becomes a promising paradigm for robotic manipulation, yet existing approaches typically support either purely image-based forecasting or reasoning over partial 3D geometry, limiting their ability to predict complete 4D scene dynamics. This work proposes a novel embodied 4D world model that enables geometrically consistent, arbitrary-view RGBD generation: given only a single-view RGBD observation as input, the model imagines the remaining viewpoints, which can then be back-projected and fused to assemble a more complete 3D structure across time. To efficiently learn the multi-view, cross-modality generation, we explicitly design cross-view and cross-modality feature fusion that jointly encourage consistency between RGB and depth and enforce geometric alignment across views. Beyond prediction, converting generated futures into actions is often handled by inverse dynamics, which is ill-posed because multiple actions can explain the same transition. We address this with a test-time action optimization strategy that backpropagates through the generative model to infer a trajectory-level latent best matching the predicted future, and a residual inverse dynamics model that turns this trajectory prior into accurate executable actions. Experiments on three datasets demonstrate strong performance on both 4D scene generation and downstream manipulation, and ablations provide practical insights into the key design choices.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 37 of its 40 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

37 of 40 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025ChronoDreamer: Action-Conditioned World Model as an Online Simulator for Robotic Planning21 Dec
  2. 2025Robot Learning from a Physical World Model10 Nov
  3. 2025Geometry-aware 4D Video Generation for Robot Manipulation1 Jul
  4. 2025ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation29 Jun
  5. 2025ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model24 Jun
  6. 2025TesserAct: Learning 4D Embodied World Models29 Apr
  7. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation3 Jan
  8. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation19 Dec

Other references · 29

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  8. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  9. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  10. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  11. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  12. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
17 more
  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  2. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
  3. 2025Unified Video Action Modelcited by 195
  4. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  5. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  6. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  7. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  8. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  9. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  10. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  11. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  12. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  13. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  14. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  15. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  16. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Modelscited by 24
  17. 2024Reinforcement Learning with Generalizable Gaussian Splattingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

Similar, not cited

Close in wording, no citation either way

  1. 2026RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
  2. 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modeling
  3. 2026GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
  4. 2026PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation
  5. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation
  6. 2026Learning 4D Geometric Priors for Inference-Efficient World Action Models
  7. 2026Robotic Manipulation is Vision-to-Geometry Mapping: Vision-Geometry Backbones over Language and Video Models
  8. 2026MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation