MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World-model-based imagine-then-act becomes a promising paradigm for robotic manipulation, yet existing approaches typically support either purely image-based forecasting or reasoning over partial 3D geometry, limiting their ability to predict complete 4D scene dynamics. This work proposes a novel embodied 4D world model that enables geometrically consistent, arbitrary-view RGBD generation: given only a single-view RGBD observation as input, the model imagines the remaining viewpoints, which can then be back-projected and fused to assemble a more complete 3D structure across time. To efficiently learn the multi-view, cross-modality generation, we explicitly design cross-view and cross-modality feature fusion that jointly encourage consistency between RGB and depth and enforce geometric alignment across views. Beyond prediction, converting generated futures into actions is often handled by inverse dynamics, which is ill-posed because multiple actions can explain the same transition. We address this with a test-time action optimization strategy that backpropagates through the generative model to infer a trajectory-level latent best matching the predicted future, and a residual inverse dynamics model that turns this trajectory prior into accurate executable actions. Experiments on three datasets demonstrate strong performance on both 4D scene generation and downstream manipulation, and ablations provide practical insights into the key design choices.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 37 of its 40 references that are in the map. First posted on arXiv 10 Feb 2026.
Builds on
37 of 40 references.
Closest ancestors · 8
Same direction, newest first
- 2025ChronoDreamer: Action-Conditioned World Model as an Online Simulator for Robotic Planning21 Dec
- 2025Robot Learning from a Physical World Model10 Nov
- 2025Geometry-aware 4D Video Generation for Robot Manipulation1 Jul
- 2025ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation29 Jun
- 2025ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model24 Jun
- 2025TesserAct: Learning 4D Embodied World Models29 Apr
- 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation3 Jan
- 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation19 Dec
Other references · 29
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022Video Diffusion Modelscited by 2,280
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
- 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
17 more
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
- 2025Unified Video Action Modelcited by 195
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
- 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
- 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
- 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
- 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
- 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
- 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
- 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
- 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
- 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
- 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Modelscited by 24
- 2024Reinforcement Learning with Generalizable Gaussian Splattingcited by 11
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokenscited by 12
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026World Action Models: A Surveycited by 10
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Spacecited by 3
- 2026World Pilot: Steering Vision-Language-Action Models with World-Action Priorscited by 2
- 2026μ0: A Scalable 3D Interaction-Trace World Modelcited by 2
- 2026SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulatorscited by 1
- 2026Quo Vadis, World Modeling?cited by 0
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
1 more
- 2026RoGSW4RLD: Feed-Forward 4D Gaussian Lifting for Robot World Model Rollouts28 Sep 2026
- 2026SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators8 Sep 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space2 Aug 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokens12 Jun 2026
- 2026μ0: A Scalable 3D Interaction-Trace World Model11 Jun 2026
- 2026World Pilot: Steering Vision-Language-Action Models with World-Action Priors10 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026World Action Models: The Next Frontier in Embodied AI12 May 2026
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr 2026
1 more
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokenscited by 12
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026World Action Models: A Surveycited by 10
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Spacecited by 3
- 2026World Pilot: Steering Vision-Language-Action Models with World-Action Priorscited by 2
- 2026μ0: A Scalable 3D Interaction-Trace World Modelcited by 2
- 2026SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulatorscited by 1
- 2026Quo Vadis, World Modeling?cited by 0
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
12 most cited of 13 All 13 (JSON)
- 2026RoGSW4RLD: Feed-Forward 4D Gaussian Lifting for Robot World Model Rollouts28 Sep 2026
- 2026SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators8 Sep 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space2 Aug 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokens12 Jun 2026
- 2026μ0: A Scalable 3D Interaction-Trace World Model11 Jun 2026
- 2026World Pilot: Steering Vision-Language-Action Models with World-Action Priors10 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026World Action Models: The Next Frontier in Embodied AI12 May 2026
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr 2026
12 newest of 13 All 13 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
- 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modeling
- 2026GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
- 2026PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation
- 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation
- 2026Learning 4D Geometric Priors for Inference-Efficient World Action Models
- 2026Robotic Manipulation is Vision-to-Geometry Mapping: Vision-Geometry Backbones over Language and Video Models
- 2026MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation