mlsanityData to 14 Oct 2026

Do Agents Dream of Electric Sheep?: Improving Generalization in Reinforcement Learning through Generative Learning

arXiv 2403.07979 ↗First posted 12 Mar 2024cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

The Overfitted Brain hypothesis suggests dreams happen to allow generalization in the human brain. Here, we ask if the same is true for reinforcement learning agents as well. Given limited experience in a real environment, we use imagination-based reinforcement learning to train a policy on dream-like episodes, where non-imaginative, predicted trajectories are modified through generative augmentations. Experiments on four ProcGen environments show that, compared to classic imagination and offline training on collected experience, our method can reach a higher level of generalization when dealing with sparsely rewarded environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 47 of its 63 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

47 of 63 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  9. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  10. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  12. 2023Mastering Diverse Domains through World Modelscited by 1,253
35 more
  1. 2017Mixed Precision Trainingcited by 1,156
  2. 2020Mastering Atari with Discrete World Modelscited by 1,007
  3. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  5. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2019Model-Based Reinforcement Learning for Ataricited by 502
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  9. 2020Reinforcement Learning with Augmented Datacited by 468
  10. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  11. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  12. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  13. 2022Transformers are Sample-Efficient World Modelscited by 328
  14. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  15. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  16. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  17. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  18. 2020Prioritized Level Replaycited by 198
  19. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  20. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  21. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 140
  22. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  23. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  24. 2018Generalization and Regularization in DQNcited by 118
  25. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  26. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  27. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  28. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  29. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  30. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
  31. 2021Generalized Proximal Policy Optimization with Sample Reusecited by 40
  32. 2017Robust Locally-Linear Controllable Embeddingcited by 38
  33. 2020Rotation, Translation, and Cropping for Zero-Shot Generalizationcited by 24
  34. 2020The Overfitted Brain: Dreams evolved to assist generalizationcited by 16
  35. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learningcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GIRL: Generative Imagination Reinforcement Learning via Information-Theoretic Hallucination Control
  2. 2021Dropout's Dream Land: Generalization from Learned Simulators to Reality
  3. 2022Towards biologically plausible Dreaming and Planning in recurrent spiking networks
  4. 2018World Models
  5. 2024Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming
  6. 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination
  7. 2026Probabilistic Dreaming for World Models
  8. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction