# MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation

arXiv: https://arxiv.org/abs/2309.14236 · HTML: https://mlsanity.com/paper/2309.14236/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 30 of 38)

- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control](https://mlsanity.com/paper/1812.00568/) (2018)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [Sim-to-Real Robot Learning from Pixels with Progressive Nets](https://mlsanity.com/paper/1610.04286/) (2016)
- [RRL: Resnet as representation for Reinforcement Learning](https://mlsanity.com/paper/2107.03380/) (2021)
- [Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost](https://mlsanity.com/paper/1810.06045/) (2018)
- [Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials](https://mlsanity.com/paper/2210.05178/) (2022)

## Built on by (most cited first, 28 of 28)

- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [A Survey: Learning Embodied Intelligence from Physical Simulators and World Models](https://mlsanity.com/paper/2507.00917/) (2025)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [RISE: Self-Improving Robot Policy with Compositional World Model](https://mlsanity.com/paper/2602.11075/) (2026)
- [Visuo-Tactile World Models](https://mlsanity.com/paper/2602.06001/) (2026)
- [Hierarchical World Models as Visual Whole-Body Humanoid Controllers](https://mlsanity.com/paper/2405.18418/) (2024)
- [Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own](https://mlsanity.com/paper/2310.02635/) (2023)
- [Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning](https://mlsanity.com/paper/2503.01837/) (2025)
- [World Models for General Surgical Grasping](https://mlsanity.com/paper/2405.17940/) (2024)
- [Empowering Multi-Robot Cooperation via Sequential World Models](https://mlsanity.com/paper/2509.13095/) (2025)
- [The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2411.10175/) (2024)
- [Bisimulation metric for Model Predictive Control](https://mlsanity.com/paper/2410.04553/) (2024)
- [Overcoming Statistical Bias in Action-Controllable World Models](https://mlsanity.com/paper/2608.04653/) (2026)
- [Dual-Mind World Models: A General Framework for Learning in Dynamic Wireless Networks](https://mlsanity.com/paper/2510.24546/) (2025)
- [Representing Positional Information in Generative World Models for Object Manipulation](https://mlsanity.com/paper/2409.12005/) (2024)
- [Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/2505.04619/) (2025)
- [Situationally-Aware Dynamics Learning](https://mlsanity.com/paper/2505.19574/) (2025)
- [Visuomotor Grasping with World Models for Surgical Robots](https://mlsanity.com/paper/2508.11200/) (2025)
- [ForeDiffusion: Foresight-Conditioned Diffusion Policy via Future View Construction for Robot Manipulation](https://mlsanity.com/paper/2601.12925/) (2026)
- [SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning](https://mlsanity.com/paper/2602.03201/) (2026)
- [Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination](https://mlsanity.com/paper/2605.04568/) (2026)
- [CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2609.08853/) (2026)
- [Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditions](https://mlsanity.com/paper/2409.17896/) (2024)
- [Real-World Robot Control by Deep Active Inference With a Temporally Hierarchical World Model](https://mlsanity.com/paper/2512.01924/) (2025)
- [Knowledge-Guided Manipulation Using Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2603.24083/) (2026)
- [WM-Craftnet: World Synesthesia Model for Generalizable and Robust Dexterous In-Hand Manipulation](https://mlsanity.com/paper/2609.07002/) (2026)
- [Generalizable Robotic Insertion with World Models](https://mlsanity.com/paper/2609.28258/) (2026)

## Abstract (arXiv)

Robotic systems that aspire to operate in uninstrumented real-world environments must perceive the world directly via onboard sensing. Vision-based learning systems aim to eliminate the need for environment instrumentation by building an implicit understanding of the world based on raw pixels, but navigating the contact-rich high-dimensional search space from solely sparse visual reward signals significantly exacerbates the challenge of exploration. The applicability of such systems is thus typically restricted to simulated or heavily engineered environments since agent exploration in the real-world without the guidance of explicit state estimation and dense rewards can lead to unsafe behavior and safety faults that are catastrophic. In this study, we isolate the root causes behind these limitations to develop a system, called MoDem-V2, capable of learning contact-rich manipulation directly in the uninstrumented real world. Building on the latest algorithmic advancements in model-based reinforcement learning (MBRL), demo-bootstrapping, and effective exploration, MoDem-V2 can acquire contact-rich dexterous manipulation skills directly in the real world. We identify key ingredients for leveraging demonstrations in model learning while respecting real-world safety considerations -- exploration centering, agency handover, and actor-critic ensembles. We empirically demonstrate the contribution of these ingredients in four complex visuo-motor manipulation problems in both simulation and the real world. To the best of our knowledge, our work presents the first successful system for demonstration-augmented visual MBRL trained directly in the real world. Visit https://sites.google.com/view/modem-v2 for videos and more details.