mlsanityData to 14 Oct 2026

DayDreamer: World Models for Physical Robot Learning

arXiv 2206.14176 ↗First posted 28 Jun 2022cs.RO cs.AI cs.LGcited by 400

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

To solve tasks in complex environments, robots need to learn from experience. Deep reinforcement learning is a common approach to robot learning but requires a large amount of trial and error to learn, limiting its deployment in the physical world. As a consequence, many advances in robot learning rely on simulators. On the other hand, learning inside of simulators fails to capture the complexity of the real world, is prone to simulator inaccuracies, and the resulting behaviors do not adapt to changes in the world. The Dreamer algorithm has recently shown great promise for learning from small amounts of interaction by planning within a learned world model, outperforming pure reinforcement learning in video games. Learning a world model to predict the outcomes of potential actions enables planning in imagination, reducing the amount of trial and error needed in the real environment. However, it is unknown whether Dreamer can facilitate faster learning on physical robots. In this paper, we apply Dreamer to 4 robots to learn online and directly in the real world, without simulators. Dreamer trains a quadruped robot to roll off its back, stand up, and walk from scratch and without resets in only 1 hour. We then push the robot and find that Dreamer adapts within 10 minutes to withstand perturbations or quickly roll over and stand back up. On two different robotic arms, Dreamer learns to pick and place multiple objects directly from camera images and sparse rewards, approaching human performance. On a wheeled robot, Dreamer learns to navigate to a goal position purely from camera images, automatically resolving ambiguity about the robot orientation. Using the same hyperparameters across all experiments, we find that Dreamer is capable of online learning in the real world, establishing a strong baseline. We release our infrastructure for future applications of world models to robot learning.

Key facts. Cited by 400 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 57 of its 74 references that are in the map. First posted on arXiv 28 Jun 2022.

Builds on

57 of 74 references.

Closest ancestors · 8

Same direction, newest first

  1. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations27 Oct
  2. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning24 Sep
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction29 Jul
  5. 2020Planning to Explore via Self-Supervised World Models12 May
  6. 2020Learning to Walk in the Real World with Minimal Human Effort20 Feb
  7. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May

Other references · 49

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  10. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  12. 2019Solving Rubik's Cube with a Robot Handcited by 734
37 more
  1. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  2. 2018Learning Dexterous In-Hand Manipulationcited by 687
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  4. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  5. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  6. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  7. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  8. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  9. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  10. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  11. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  12. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  13. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  14. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  15. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  16. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  17. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
  18. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  19. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  20. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  21. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
  22. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  23. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  24. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 81
  25. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  26. 2019Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewardscited by 64
  27. 2019Data Efficient Reinforcement Learning for Legged Robotscited by 63
  28. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 58
  29. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
  30. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  31. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 47
  32. 2022Safe Reinforcement Learning for Legged Locomotioncited by 35
  33. 2019Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresightcited by 34
  34. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 30
  35. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  36. 2015Adapting Deep Visuomotor Representations with Weak Pairwise Constraintscited by 25
  37. 2020Learning to Fly via Deep Model-Based Reinforcement Learningcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 65; 2024: 81; 2025: 114; 2026: 131.

400 papers.

Sort:

By area:

  1. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  3. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  4. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  5. 2026Causal World Modeling for Robot Controlcited by 321
  6. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  7. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  9. 2024Navigation World Modelscited by 272
  10. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  11. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  12. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
38 more
  1. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  2. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
  3. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  4. 2023Extreme Parkour with Legged Robotscited by 164
  5. 2023Structured World Models from Human Videoscited by 147
  6. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 138
  7. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
  8. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 136
  9. 2025Video World Models with Long-term Spatial Memorycited by 136
  10. 2025Training Agents Inside of Scalable World Modelscited by 136
  11. 2025Unified Vision-Language-Action Modelcited by 134
  12. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  13. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 124
  14. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  15. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 112
  16. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  17. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  18. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
  19. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
  20. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 100
  21. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
  22. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 89
  23. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  24. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  25. 2024World Models for Autonomous Driving: An Initial Surveycited by 80
  26. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 80
  27. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 68
  28. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  29. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 67
  30. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  31. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  32. 2025Long-Context State-Space Video World Modelscited by 64
  33. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  34. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  35. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)cited by 52
  36. 2025From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignmentcited by 52
  37. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticscited by 51
  38. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51

50 most cited of 400 All 400 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2018Learning Real-World Robot Policies by Dreaming
  2. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
  3. 2025Multimodal Dreaming: A Global Workspace Approach to World Model-Based Reinforcement Learning
  4. 2026DreamFormer: Dream Imitation with a Transformer World Model for Language-Conditioned Robotic Manipulation
  5. 2023Mastering Diverse Domains through World Models
  6. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  7. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models
  8. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learning