mlsanityData to 14 Oct 2026

MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation

arXiv 2309.14236 ↗First posted 25 Sep 2023cs.RO cs.AI cs.CVcited by 28

On the map: World models → Embodied and robot world models

Abstract (arXiv)

Robotic systems that aspire to operate in uninstrumented real-world environments must perceive the world directly via onboard sensing. Vision-based learning systems aim to eliminate the need for environment instrumentation by building an implicit understanding of the world based on raw pixels, but navigating the contact-rich high-dimensional search space from solely sparse visual reward signals significantly exacerbates the challenge of exploration. The applicability of such systems is thus typically restricted to simulated or heavily engineered environments since agent exploration in the real-world without the guidance of explicit state estimation and dense rewards can lead to unsafe behavior and safety faults that are catastrophic. In this study, we isolate the root causes behind these limitations to develop a system, called MoDem-V2, capable of learning contact-rich manipulation directly in the uninstrumented real world. Building on the latest algorithmic advancements in model-based reinforcement learning (MBRL), demo-bootstrapping, and effective exploration, MoDem-V2 can acquire contact-rich dexterous manipulation skills directly in the real world. We identify key ingredients for leveraging demonstrations in model learning while respecting real-world safety considerations -- exploration centering, agency handover, and actor-critic ensembles. We empirically demonstrate the contribution of these ingredients in four complex visuo-motor manipulation problems in both simulation and the real world. To the best of our knowledge, our work presents the first successful system for demonstration-augmented visual MBRL trained directly in the real world. Visit https://sites.google.com/view/modem-v2 for videos and more details.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 38 of its 47 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

38 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb

Other references · 37

  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  11. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  12. 2019Model-Based Reinforcement Learning for Ataricited by 502
25 more
  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  5. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  6. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  7. 2021Mastering Atari Games with Limited Datacited by 237
  8. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  9. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  10. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  11. 2019Active Domain Randomizationcited by 170
  12. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  13. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  14. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  15. 2021RRL: Resnet as representation for Reinforcement Learningcited by 91
  16. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 84
  17. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  18. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 72
  19. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  20. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  21. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  22. 2020Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learningcited by 23
  23. 2017Learning to Run with Actor-Critic Ensemblecited by 8
  24. 2022Motivating Physical Activity via Competitive Human-Robot Interactioncited by 3
  25. 2023Stackelberg Games for Learning Emergent Behaviors During Competitive Autocurriculacited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 9; 2026: 11.

28 papers.

Sort:

By area:

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  3. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
  4. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  5. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 40
  6. 2026Visuo-Tactile World Modelscited by 29
  7. 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllerscited by 27
  8. 2023Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Owncited by 19
  9. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learningcited by 11
  10. 2024World Models for General Surgical Graspingcited by 9
  11. 2025Empowering Multi-Robot Cooperation via Sequential World Modelscited by 9
  12. 2024The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learningcited by 7
16 more
  1. 2024Bisimulation metric for Model Predictive Controlcited by 6
  2. 2026Overcoming Statistical Bias in Action-Controllable World Modelscited by 6
  3. 2025Dual-Mind World Models: A General Framework for Learning in Dynamic Wireless Networkscited by 4
  4. 2024Representing Positional Information in Generative World Models for Object Manipulationcited by 3
  5. 2025Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulationcited by 2
  6. 2025Situationally-Aware Dynamics Learningcited by 1
  7. 2025Visuomotor Grasping with World Models for Surgical Robotscited by 1
  8. 2026ForeDiffusion: Foresight-Conditioned Diffusion Policy via Future View Construction for Robot Manipulationcited by 1
  9. 2026SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learningcited by 1
  10. 2026Dream-MPC: Gradient-Based Model Predictive Control with Latent Imaginationcited by 1
  11. 2026CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learningcited by 1
  12. 2024Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditionscited by 0
  13. 2025Real-World Robot Control by Deep Active Inference With a Temporally Hierarchical World Modelcited by 0
  14. 2026Knowledge-Guided Manipulation Using Multi-Task Reinforcement Learningcited by 0
  15. 2026WM-Craftnet: World Synesthesia Model for Generalizable and Robust Dexterous In-Hand Manipulationcited by 0
  16. 2026Generalizable Robotic Insertion with World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich Manipulation
  2. 2025Safe Learning for Contact-Rich Robot Tasks: A Survey from Classical Learning-Based Methods to Safe Foundation Models
  3. 2026World Models for Robotic Manipulation: A Survey
  4. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models
  5. 2026Beyond Pixels: Learning Invariant Rewards for Real-World Robotics From a Few Demonstrations
  6. 2026Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
  7. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  8. 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation