mlsanityData to 14 Oct 2026

Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learning

arXiv 2103.06386 ↗First posted 10 Mar 2021cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Meta-reinforcement learning typically requires orders of magnitude more samples than single task reinforcement learning methods. This is because meta-training needs to deal with more diverse distributions and train extra components such as context encoders. To address this, we propose a novel self-supervised learning task, which we named Trajectory Contrastive Learning (TCL), to improve meta-training. TCL adopts contrastive learning and trains a context encoder to predict whether two transition windows are sampled from the same trajectory. TCL leverages the natural hierarchical structure of context-based meta-RL and makes minimal assumptions, allowing it to be generally applicable to context-based meta-RL algorithms. It accelerates the training of context encoders and improves meta-training overall. Experiments show that TCL performs better or comparably than a strong meta-RL baseline in most of the environments on both meta-RL MuJoCo (5 of 6) and Meta-World benchmarks (44 out of 50).

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

33 of 49 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2015Trust Region Policy Optimizationcited by 2,580
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
21 more
  1. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  2. 2018Learning Dexterous In-Hand Manipulationcited by 687
  3. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  4. 2016Learning to learn by gradient descent by gradient descentcited by 620
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  6. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  8. 2016Learning to reinforcement learncited by 377
  9. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  10. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  11. 2016Learning to Optimizecited by 195
  12. 2016Towards a Neural Statisticiancited by 121
  13. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  14. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  15. 2016One-Shot Generalization in Deep Generative Modelscited by 45
  16. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 44
  17. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  18. 2018Learning Actionable Representations from Visual Observationscited by 35
  19. 2020Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcited by 29
  20. 2019Learning Internal Representations (COLT 1995)cited by 25
  21. 2018Learning to Explore with Meta-Policy Gradientcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 1; 2026: 1.

9 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  3. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  4. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  5. 2023Contrastive learning-based agent modeling for deep reinforcement learningcited by 2
  6. 2022Decomposed Mutual Information Optimization for Generalized Context in Meta-Reinforcement Learningcited by 1
  7. 2024Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learningcited by 1
  8. 2024Memory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agentscited by 1
  9. 2026In-Context Reinforcement Learning under Non-Stationarity: A Surveycited by 1