# Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination

arXiv: https://arxiv.org/abs/2311.06673 · HTML: https://mlsanity.com/paper/2311.06673/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 60 references that are in the map.
- First posted on arXiv 11 Nov 2023.

## Builds on (most cited first, 30 of 35)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Meta-Reinforcement Learning of Structured Exploration Strategies](https://mlsanity.com/paper/1802.07245/) (2018)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Label-Free Supervision of Neural Networks with Physics and Domain Knowledge](https://mlsanity.com/paper/1609.05566/) (2016)
- [Understanding data augmentation for classification: when to warp?](https://mlsanity.com/paper/1609.08764/) (2016)
- [Improving Generalization in Meta Reinforcement Learning using Learned Objectives](https://mlsanity.com/paper/1910.04098/) (2019)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Improving Generalization in Meta-learning via Task Augmentation](https://mlsanity.com/paper/2007.13040/) (2020)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [Data Augmentation for Meta-Learning](https://mlsanity.com/paper/2010.07092/) (2020)
- [Model-based Adversarial Meta-Reinforcement Learning](https://mlsanity.com/paper/2006.08875/) (2020)
- [DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction](https://mlsanity.com/paper/2203.00494/) (2022)
- [Guided Meta-Policy Search](https://mlsanity.com/paper/1904.00956/) (2019)
- [Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling](https://mlsanity.com/paper/2006.07178/) (2020)
- [Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models](https://mlsanity.com/paper/2006.10236/) (2020)
- [Beating Atari with Natural Language Guided Reinforcement Learning](https://mlsanity.com/paper/1704.05539/) (2017)
- [A theory of independent mechanisms for extrapolation in generative models](https://mlsanity.com/paper/2004.00184/) (2020)
- [Physics-informed deep generative models](https://mlsanity.com/paper/1812.03511/) (2018)

## Built on by (most cited first, 6 of 6)

- [Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization](https://mlsanity.com/paper/2403.10967/) (2024)
- [Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning](https://mlsanity.com/paper/2404.09521/) (2024)
- [Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks](https://mlsanity.com/paper/2502.02834/) (2025)
- [Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention](https://mlsanity.com/paper/2603.14483/) (2026)
- [cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending](https://mlsanity.com/paper/2508.20818/) (2025)
- [Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation](https://mlsanity.com/paper/2609.20300/) (2026)

## Abstract (arXiv)

Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art algorithms require the meta-training tasks to have a dense coverage on the task distribution and a great amount of data for each of them. In this paper, we propose MetaDreamer, a context-based Meta RL algorithm that requires less real training tasks and data by doing meta-imagination and MDP-imagination. We perform meta-imagination by interpolating on the learned latent context space with disentangled properties, as well as MDP-imagination through the generative world model where physical knowledge is added to plain VAE networks. Our experiments with various benchmarks show that MetaDreamer outperforms existing approaches in data efficiency and interpolated generalization.