mlsanityData to 7 Oct 2026

DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

arXiv 2203.00494 ↗First posted 1 Mar 2022cs.LG cs.AI cs.SYcited by 29

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pixels that uses discrete world models to represent latent states with categorical variables. Dreaming is also a form of reinforcement learning from pixels that attempts to avoid the autoencoding process in general world model training by involving a reconstruction-free contrastive learning objective. The proposed DreamingV2 is a novel approach of adopting both the discrete representation of DreamingV2 and the reconstruction-free objective of Dreaming. Compared to DreamerV2 and other recent model-based methods without reconstruction, DreamingV2 achieves the best scores on five simulated challenging 3D robot arm tasks. We believe that DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments, and the reconstruction-free fashion well manages complex vision observations.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 33 of its 36 references that are in the map. First posted on arXiv 1 Mar 2022.

Builds on

33 of 36 references.

Closest ancestors · 7

Same direction, newest first

  1. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations27 Oct
  2. 2021Multi-Modal Mutual Information (MuMMI) Training for Robust Self-Supervised Deep Reinforcement Learning6 Jul
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction29 Jul
  5. 2020Planning to Explore via Self-Supervised World Models12 May
  6. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  7. 2018World Models27 Mar

Other references · 26

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  11. 2018DeepMind Control Suitecited by 935
  12. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
14 more
  1. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  2. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  4. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  5. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  6. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  7. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
  8. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 46
  9. 2019Variational Inference MPC for Bayesian Model-based Reinforcement Learningcited by 39
  10. 2019Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Modelscited by 26
  11. 2020PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inferencecited by 22
  12. 2020Contrastive Variational Reinforcement Learning for Complex Observationscited by 15
  13. 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
  14. 20173DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Cloudscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 5; 2025: 9; 2026: 9.

29 papers.

Sort:

By area:

  1. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  2. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
  3. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 70
  4. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  5. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  6. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  7. 2023World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challengescited by 35
  8. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  9. 2025Embodied AI: From LLMs to World Modelscited by 23
  10. 2026Predictive but Not Plannable: RC-aux for Latent World Modelscited by 16
  11. 2024Generative Emergent Communication: Large Language Model is a Collective World Modelcited by 15
  12. 2024MuDreamer: Learning Predictive World Models without Reconstructioncited by 13
17 more
  1. 2025Bootstrapped Model Predictive Controlcited by 13
  2. 2025Four Principles for Physically Interpretable World Modelscited by 10
  3. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  4. 2023Combining Reconstruction and Contrastive Methods for Multimodal Representations in RLcited by 9
  5. 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imaginationcited by 6
  6. 2026Enhancing Policy Learning with World-Action Modelcited by 6
  7. 2026MRI Contrast Enhancement Kinetics World Modelcited by 4
  8. 2026Next Embedding Prediction Makes World Models Strongercited by 4
  9. 2022Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertioncited by 3
  10. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  11. 2026A Definition and Roadmap for World Modelscited by 3
  12. 2025Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agentscited by 2
  13. 2026Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Predictioncited by 2
  14. 2024Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learningcited by 1
  15. 2023World-Model-Based Control for Industrial box-packing of Multiple Objects using NewtonianVAEcited by 0
  16. 2025MInCo: Mitigating Information Conflicts in Distracted Visual Model-based Reinforcement Learningcited by 0
  17. 2026CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Learning Real-World Robot Policies by Dreaming
  2. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learning
  3. 2022DayDreamer: World Models for Physical Robot Learning
  4. 2025Multimodal Dreaming: A Global Workspace Approach to World Model-Based Reinforcement Learning
  5. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models
  6. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models
  7. 2025Zero-shot World Models via Search in Memory
  8. 2023Model-Based Reinforcement Learning with Isolated Imaginations