mlsanityData to 7 Oct 2026

Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks

arXiv 2502.02834 ↗First posted 5 Feb 2025cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Meta reinforcement learning aims to develop policies that generalize to unseen tasks sampled from a task distribution. While context-based meta-RL methods improve task representation using task latents, they often struggle with out-of-distribution (OOD) tasks. To address this, we propose Task-Aware Virtual Training (TAVT), a novel algorithm that accurately captures task characteristics for both training and OOD scenarios using metric-based representation learning. Our method successfully preserves task characteristics in virtual tasks and employs a state regularization technique to mitigate overestimation errors in state-varying environments. Numerical results demonstrate that TAVT significantly enhances generalization to OOD tasks across various MuJoCo and MetaWorld environments. Our code is available at https://github.com/JM-Kim-94/tavt.git.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 49 of its 66 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

49 of 66 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2017Improved Training of Wasserstein GANscited by 2,223
  6. 2016OpenAI Gymcited by 1,684
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  11. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
37 more
  1. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  2. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  3. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  4. 2017Meta Learning Shared Hierarchiescited by 93
  5. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  6. 2019Meta-Q-Learningcited by 62
  7. 2019Continuous Meta-Learning without Taskscited by 50
  8. 2022Transformers are Meta-Reinforcement Learnerscited by 39
  9. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  10. 2022Skill-based Meta-Reinforcement Learningcited by 36
  11. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  12. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  13. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  14. 2020Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcited by 29
  15. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  16. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  17. 2021Block Contextual MDPs for Continual Learningcited by 25
  18. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 24
  19. 2023Context Shift Reduction for Offline Meta-Reinforcement Learningcited by 17
  20. 2019Meta Reinforcement Learning with Task Embedding and Shared Policycited by 15
  21. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  22. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  23. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 13
  24. 2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
  25. 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
  26. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  27. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  28. 2020Curriculum in Gradient-Based Meta-Reinforcement Learningcited by 9
  29. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  30. 2023Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitationscited by 9
  31. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  32. 2024Exclusively Penalized Q-learning for Offline Reinforcement Learningcited by 8
  33. 2022Meta-Learning Parameterized Skillscited by 7
  34. 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imaginationcited by 6
  35. 2021Hindsight Foresight Relabeling for Meta-Reinforcement Learningcited by 4
  36. 2022Efficient Meta Reinforcement Learning for Preference-based Fast Adaptationcited by 4
  37. 2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skillscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  2. 2025An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learningcited by 0
  3. 2026Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learningcited by 0