Mask World Model: Predicting What Matters for Robust Robot Policy Learning
On the map: World models → Embodied and robot world models
also in 1 more
Abstract (arXiv)
World models derived from large-scale video generative pre-training have emerged as a promising paradigm for generalist robot policy learning. However, standard approaches often focus on high-fidelity RGB video prediction, this can result in overfitting to irrelevant factors, such as dynamic backgrounds and illumination changes. These distractions reduce the model's ability to generalize, ultimately leading to unreliable and fragile control policies. To address this, we introduce the Mask World Model (MWM), which leverages video diffusion architectures to predict the evolution of semantic masks instead of pixels. This shift imposes a geometric information bottleneck, forcing the model to capture essential physical dynamics and contact relations while filtering out visual noise. We seamlessly integrate this mask dynamics backbone with a diffusion-based policy head to enable robust end-to-end control. Extensive evaluations demonstrate the superiority of MWM on the LIBERO and RLBench simulation benchmarks, significantly outperforming the state-of-the-art RGB-based world models. Furthermore, real-world experiments and robustness evaluation (via random token pruning) reveal that MWM exhibits superior generalization capabilities and robust resilience to texture information loss.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 22 of its 35 references that are in the map. First posted on arXiv 21 Apr 2026.
Builds on
22 of 35 references.
Closest ancestors · 7
Same direction, newest first
- 2026TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion26 Jan
- 2025WristWorld: Generating Wrist-Views via 4D World Models for Robotic Manipulation8 Oct
- 2025MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulation26 Sep
- 2025MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis23 Jun
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
- 2022Masked World Models for Visual Control28 Jun
Other references · 15
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
- 2025Training Agents Inside of Scalable World Modelscited by 122
- 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 80
- 2025MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulationcited by 69
- 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 66
- 2024EVA: An Embodied World Model for Future Video Anticipationcited by 28
- 2025SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulationcited by 20
- 2025Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modelingcited by 13
3 more
- 2025TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Modelscited by 12
- 2025Can World Models Benefit VLMs for World Dynamics?cited by 9
- 2025Disentangled World Models: Learning to Transfer Semantic Knowledge from Distracting Videos for Reinforcement Learningcited by 6
Built on by
Citing papers per year: 2026: 15.
15 papers.
Sort:
By area:
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulationcited by 17
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 15
- 2026World Action Models: A Surveycited by 10
- 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shiftscited by 10
- 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AIcited by 8
- 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
- 2026Foresight Without Seeing: Latent Futures for World Action Modelscited by 4
- 2026DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulationcited by 3
- 2026Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Modelscited by 2
- 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Modelscited by 2
- 2026SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Modelscited by 2
- 2026Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation30 Sep 2026
- 2026CAER: Causal Action Effect Reweighting for World Model Training31 Aug 2026
- 2026Foresight Without Seeing: Latent Futures for World Action Models12 Aug 2026
- 2026SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models9 Aug 2026
- 2026AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models7 Aug 2026
- 2026DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation1 Aug 2026
- 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts31 Jul 2026
- 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models28 Jul 2026
- 2026Data Pyramid for Embodied Manipulation: A Survey27 Jul 2026
- 2026Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models5 Jul 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI15 Jun 2026
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulationcited by 17
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 15
- 2026World Action Models: A Surveycited by 10
- 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shiftscited by 10
- 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AIcited by 8
- 2026Foresight Without Seeing: Latent Futures for World Action Modelscited by 4
- 2026DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulationcited by 3
- 2026Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Modelscited by 2
- 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Modelscited by 2
- 2026SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Modelscited by 2
- 2026CAER: Causal Action Effect Reweighting for World Model Trainingcited by 0
12 most cited of 13 All 13 (JSON)
- 2026Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation30 Sep 2026
- 2026CAER: Causal Action Effect Reweighting for World Model Training31 Aug 2026
- 2026Foresight Without Seeing: Latent Futures for World Action Models12 Aug 2026
- 2026SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models9 Aug 2026
- 2026DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation1 Aug 2026
- 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts31 Jul 2026
- 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models28 Jul 2026
- 2026Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models5 Jul 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI15 Jun 2026
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models11 Jun 2026
- 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation7 Jun 2026
12 newest of 13 All 13 (JSON)
- 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
- 2026Data Pyramid for Embodied Manipulation: A Survey27 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Masked Visual Actions for Unified World Modeling
- 2026Causal World Modeling for Robot Control
- 2025WorldGym: World Model as An Environment for Policy Evaluation
- 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- 2026WorldDiT: A Unified Diffusion Architecture for World and Action Modeling
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model
- 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models
- 2026Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning