mlsanityData to 7 Oct 2026

Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning

arXiv 2305.00286 ↗First posted 29 Apr 2023cs.LG cs.ROcited by 10

Not on the map. Why?

Abstract (arXiv)

Meta-reinforcement learning enables artificial agents to learn from related training tasks and adapt to new tasks efficiently with minimal interaction data. However, most existing research is still limited to narrow task distributions that are parametric and stationary, and does not consider out-of-distribution tasks during the evaluation, thus, restricting its application. In this paper, we propose MoSS, a context-based Meta-reinforcement learning algorithm based on Self-Supervised task representation learning to address this challenge. We extend meta-RL to broad non-parametric task distributions which have never been explored before, and also achieve state-of-the-art results in non-stationary and out-of-distribution tasks. Specifically, MoSS consists of a task inference module and a policy module. We utilize the Gaussian mixture model for task representation to imitate the parametric and non-parametric task variations. Additionally, our online adaptation strategy enables the agent to react at the first sight of a task change, thus being applicable in non-stationary tasks. MoSS also exhibits strong generalization robustness in out-of-distributions tasks which benefits from the reliable and robust task representation. The policy is built on top of an off-policy RL algorithm and the entire network is trained completely off-policy to ensure high sample efficiency. On MuJoCo and Meta-World benchmarks, MoSS outperforms prior works in terms of asymptotic performance, sample efficiency (3-50x faster), adaptation efficiency, and generalization robustness on broad and diverse task distributions.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 29 Apr 2023.

Builds on

28 of 43 references.

References · 28

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  9. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  10. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  11. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  12. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
16 more
  1. 2016Variational Deep Embedding: An Unsupervised and Generative Approach to Clusteringcited by 219
  2. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  3. 2019Continual Unsupervised Representation Learningcited by 147
  4. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  5. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  6. 2019Meta reinforcement learning as task inferencecited by 75
  7. 2019Meta-Q-Learningcited by 62
  8. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 43
  9. 2020Exploration in Approximate Hyper-State Space for Meta Reinforcement Learningcited by 28
  10. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 24
  11. 2020MELD: Meta-Reinforcement Learning from Images via Latent State Modelscited by 22
  12. 2022Meta-Reinforcement Learning via Language Instructionscited by 18
  13. 2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
  14. 2021Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learningcited by 8
  15. 2020Complex Robotic Manipulation via Graph-Based Hindsight Goal Generationcited by 6
  16. 2022Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 2; 2026: 3.

10 papers.

  1. 2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 12
  2. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  3. 2023Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challengescited by 2
  4. 2026Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustnesscited by 2
  5. 2023Contact Energy Based Hindsight Experience Prioritizationcited by 1
  6. 2023On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQRcited by 1
  7. 2024Memory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agentscited by 1
  8. 2024Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learningcited by 1
  9. 2026DADP: Domain Adaptive Diffusion Policycited by 1
  10. 2026Knowledge Reutilization in Meta-Reinforcement Learningcited by 0