mlsanityData to 14 Oct 2026

Masked World Models for Visual Control

arXiv 2206.14244 ↗First posted 28 Jun 2022cs.RO cs.AI cs.CVcited by 170

On the map: World models → Embodied and robot world models

Abstract (arXiv)

Visual model-based reinforcement learning (RL) has the potential to enable sample-efficient robot learning from visual observations. Yet the current approaches typically train a single model end-to-end for learning both visual representations and dynamics, making it difficult to accurately model the interaction between robots and small objects. In this work, we introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. Specifically, we train an autoencoder with convolutional layers and vision transformers (ViT) to reconstruct pixels given masked convolutional features, and learn a latent dynamics model that operates on the representations from the autoencoder. Moreover, to encode task-relevant information, we introduce an auxiliary reward prediction objective for the autoencoder. We continually update both autoencoder and dynamics model using online samples collected from environment interaction. We demonstrate that our decoupling approach achieves state-of-the-art performance on a variety of visual robotic tasks from Meta-world and RLBench, e.g., we achieve 81.7% success rate on 50 visual robotic manipulation tasks from Meta-world, while the baseline achieves 67.9%. Code is available on the project website: https://sites.google.com/view/mwm-rl.

Key facts. Cited by 170 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 79 of its 88 references that are in the map. First posted on arXiv 28 Jun 2022.

Builds on

79 of 88 references.

References · 79

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2021ViViT: A Video Vision Transformercited by 1,620
38 more
  1. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  2. 2018World Modelscited by 1,513
  3. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  7. 2020Mastering Atari with Discrete World Modelscited by 1,007
  8. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  9. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  10. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  11. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  12. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  13. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  14. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  15. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  16. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  17. 2019Model-Based Reinforcement Learning for Ataricited by 502
  18. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  19. 2020Reinforcement Learning with Augmented Datacited by 468
  20. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  21. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  22. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  23. 2021Early Convolutions Help Transformers See Bettercited by 396
  24. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  25. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  26. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  27. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  28. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  29. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  30. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  31. 2018DropBlock: A regularization method for convolutional networkscited by 262
  32. 2022Masked Visual Pre-training for Motor Controlcited by 248
  33. 2021Mastering Atari Games with Limited Datacited by 237
  34. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  35. 2014Efficient Object Localization Using Convolutional Networkscited by 227
  36. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  37. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  38. 2021Reinforcement Learning with Prototypical Representationscited by 179

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 48; 2024: 53; 2025: 34; 2026: 30.

170 papers.

Sort:

By area:

  1. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  2. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  3. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 339
  4. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  5. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  6. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  7. 2024Navigation World Modelscited by 272
  8. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  9. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  10. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  11. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 161
  12. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 148
38 more
  1. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  2. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  3. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  4. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  5. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  6. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
  7. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 89
  8. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  9. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  10. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70
  11. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  12. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  13. 2023Imitation Bootstrapped Reinforcement Learningcited by 64
  14. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  15. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  16. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  17. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  18. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  19. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  20. 2026Learning Latent Action World Models In The Wildcited by 47
  21. 2024Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulationcited by 46
  22. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 45
  23. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  24. 2024DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Modelscited by 44
  25. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 43
  26. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 43
  27. 2023PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observationscited by 42
  28. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 42
  29. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 39
  30. 2024Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulationcited by 38
  31. 2024Learning Manipulation by Predicting Interactioncited by 37
  32. 2025Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planningcited by 34
  33. 2023Emergent Agentic Transformer from Chain of Hindsight Experiencecited by 33
  34. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 33
  35. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  36. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 31
  37. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  38. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 26

50 most cited of 170 All 170 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  2. 2026Masked Visual Actions for Unified World Modeling
  3. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models
  4. 2026Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning
  5. 2026GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learning
  6. 2024ViSaRL: Visual Reinforcement Learning Guided by Human Saliency
  7. 2026Causal World Modeling for Robot Control
  8. 2023Language-Driven Representation Learning for Robotics