# Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2010.12142 · HTML: https://mlsanity.com/paper/2010.12142/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 9 of its 65 references that are in the map.
- First posted on arXiv 23 Oct 2020.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning](https://mlsanity.com/paper/1803.00101/) (2018)
- [Model-based Reinforcement Learning: A Survey](https://mlsanity.com/paper/2006.16712/) (2020)
- [Universal Planning Networks](https://mlsanity.com/paper/1804.00645/) (2018)
- [Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models](https://mlsanity.com/paper/1910.04142/) (2019)
- [Model-Based Planning with Discrete and Continuous Actions](https://mlsanity.com/paper/1705.07177/) (2017)

## Built on by (most cited first, 15 of 15)

- [Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model](https://mlsanity.com/paper/2210.04017/) (2022)
- [Flow-based Recurrent Belief State Learning for POMDPs](https://mlsanity.com/paper/2205.11051/) (2022)
- [On Effective Scheduling of Model-based Reinforcement Learning](https://mlsanity.com/paper/2111.08550/) (2021)
- [Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixture](https://mlsanity.com/paper/2105.13524/) (2021)
- [DMWM: Dual-Mind World Model with Long-Term Imagination](https://mlsanity.com/paper/2502.07591/) (2025)
- [The developmental trajectory of object recognition robustness: children are like small adults but unlike big deep neural networks](https://mlsanity.com/paper/2205.10144/) (2022)
- [Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Models](https://mlsanity.com/paper/2604.25416/) (2026)
- [Acting upon Imagination: when to trust imagined trajectories in model based reinforcement learning](https://mlsanity.com/paper/2105.05716/) (2021)
- [Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving](https://mlsanity.com/paper/2106.12194/) (2021)
- [Model-Driven Deep Neural Network for Enhanced AoA Estimation Using 5G gNB](https://mlsanity.com/paper/2501.00009/) (2024)
- [Ego-centric Learning of Communicative World Models for Autonomous Driving](https://mlsanity.com/paper/2506.08149/) (2025)
- [ED2: Environment Dynamics Decomposition World Models for Continuous Control](https://mlsanity.com/paper/2112.02817/) (2021)
- [Model-based Reinforcement Learning with a Hamiltonian Canonical ODE Network](https://mlsanity.com/paper/2211.00942/) (2022)
- [Do Agents Dream of Electric Sheep?: Improving Generalization in Reinforcement Learning through Generative Learning](https://mlsanity.com/paper/2403.07979/) (2024)
- [Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation](https://mlsanity.com/paper/2609.20300/) (2026)

## Abstract (arXiv)

Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by performing planning on imaginary trajectories with a learned world model. However, world model learning may suffer from overfitting to training trajectories, and thus model-based value estimation and policy search will be pone to be sucked in an inferior local policy. In this paper, we propose a novel model-based reinforcement learning algorithm, called BrIdging Reality and Dream (BIRD). It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories. We demonstrate that our approach improves sample efficiency of model-based planning, and achieves state-of-the-art performance on challenging visual control benchmarks.