mlsanityData to 14 Oct 2026

On Context Distribution Shift in Task Representation Learning for Offline Meta RL

arXiv 2304.00354 ↗First posted 1 Apr 2023cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Offline Meta Reinforcement Learning (OMRL) aims to learn transferable knowledge from offline datasets to enhance the learning process for new target tasks. Context-based Reinforcement Learning (RL) adopts a context encoder to expediently adapt the agent to new tasks by inferring the task representation, and then adjusting the policy based on this inferred representation. In this work, we focus on context-based OMRL, specifically on the challenge of learning task representation for OMRL. We conduct experiments that demonstrate that the context encoder trained on offline datasets might encounter distribution shift between the contexts used for training and testing. To overcome this problem, we present a hard-sampling-based strategy to train a robust task context encoder. Our experimental findings on diverse continuous control tasks reveal that utilizing our approach yields more robust task representations and better testing performance in terms of accumulated returns compared to baseline methods. Our code is available at https://github.com/ZJLAB-AMMI/HS-OMRL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 1 Apr 2023.

Builds on

26 of 34 references.

References · 26

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2020Supervised Contrastive Learningcited by 2,829
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  8. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  10. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  12. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
14 more
  1. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  2. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  3. 2020Contrastive Learning with Hard Negative Samplescited by 628
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  5. 2020Understanding the Behaviour of Contrastive Losscited by 398
  6. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  7. 2016Learning to reinforcement learncited by 377
  8. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  9. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  10. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  11. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
  12. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
  13. 2019Meta reinforcement learning as task inferencecited by 77
  14. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
  2. 2024Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learningcited by 2
  3. 2023Learning Top-k Subtask Planning Tree based on Discriminative Representation Pre-training for Decision Makingcited by 0
  4. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0