mlsanityData to 14 Oct 2026

Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixture

arXiv 2105.13524 ↗First posted 28 May 2021cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

The generalization ability of most meta-reinforcement learning (meta-RL) methods is largely limited to test tasks that are sampled from the same distribution used to sample training tasks. To overcome the limitation, we propose Latent Dynamics Mixture (LDM) that trains a reinforcement learning agent with imaginary tasks generated from mixtures of learned latent dynamics. By training a policy on mixture tasks along with original training tasks, LDM allows the agent to prepare for unseen test tasks during training and prevents the agent from overfitting the training tasks. LDM significantly outperforms standard meta-RL methods in test returns on the gridworld navigation and MuJoCo tasks where we strictly separate the training task distribution and the test task distribution.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 46 of its 69 references that are in the map. First posted on arXiv 28 May 2021.

Builds on

46 of 69 references.

References · 46

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  8. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  9. 2020Reinforcement Learning with Augmented Datacited by 468
  10. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  11. 2016Learning to reinforcement learncited by 377
  12. 2018Quantifying Generalization in Reinforcement Learningcited by 352
34 more
  1. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  2. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  3. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  4. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  5. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  6. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
  7. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  8. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  9. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  10. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  11. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
  12. 2019Meta-Learning without Memorizationcited by 92
  13. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  14. 2019Observational Overfitting in Reinforcement Learningcited by 91
  15. 2019Self-Supervised Generalisation with Meta Auxiliary Learningcited by 88
  16. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  17. 2019Meta reinforcement learning as task inferencecited by 77
  18. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  19. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  20. 2019Improving Generalization in Meta Reinforcement Learning using Learned Objectivescited by 68
  21. 2019Meta-Q-Learningcited by 62
  22. 2020Deep Reinforcement Learning amidst Lifelong Non-Stationaritycited by 43
  23. 2018Unsupervised Meta-Learning for Reinforcement Learningcited by 39
  24. 2020Optimizing for the Future in Non-Stationary MDPscited by 37
  25. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 34
  26. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 33
  27. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 26
  28. 2020Probabilistic Active Meta-Learningcited by 23
  29. 2019Single Episode Policy Transfer in Reinforcement Learningcited by 18
  30. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learningcited by 15
  31. 2018Virtual Class Enhanced Discriminative Embedding Learningcited by 13
  32. 2020Curriculum in Gradient-Based Meta-Reinforcement Learningcited by 9
  33. 2021Linear Representation Meta-Reinforcement Learning for Instant Adaptationcited by 9
  34. 2020Provably Efficient Model-based Policy Adaptationcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2; 2024: 4; 2025: 3.

12 papers.

  1. 2022Test-Time Adaptation via Self-Training with Nearest Neighbor Informationcited by 64
  2. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 19
  3. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 14
  4. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  5. 2022Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approachcited by 6
  6. 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imaginationcited by 6
  7. 2024Towards Unraveling and Improving Generalization in World Modelscited by 4
  8. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  9. 2024Hierarchical Meta-Reinforcement Learning via Automated Macro-Action Discoverycited by 2
  10. 2024Memory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agentscited by 1
  11. 2025Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contextscited by 0
  12. 2025An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learningcited by 0