mlsanityData to 14 Oct 2026

Meta-Reinforcement Learning via Exploratory Task Clustering

arXiv 2302.07958 ↗First posted 15 Feb 2023cs.LG cs.AI cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Meta-reinforcement learning (meta-RL) aims to quickly solve new tasks by leveraging knowledge from prior tasks. However, previous studies often assume a single mode homogeneous task distribution, ignoring possible structured heterogeneity among tasks. Leveraging such structures can better facilitate knowledge sharing among related tasks and thus improve sample efficiency. In this paper, we explore the structured heterogeneity among tasks via clustering to improve meta-RL. We develop a dedicated exploratory policy to discover task structures via divide-and-conquer. The knowledge of the identified clusters helps to narrow the search space of task-specific information, leading to more sample efficient policy adaptation. Experiments on various MuJoCo tasks showed the proposed method can unravel cluster structures effectively in both rewards and state dynamics, proving strong advantages against a set of state-of-the-art baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

27 of 35 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  10. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  11. 2016Learning to reinforcement learncited by 377
  12. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
15 more
  1. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 220
  2. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  3. 2019Continual Unsupervised Representation Learningcited by 147
  4. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  5. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  6. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  7. 2019Hierarchically Structured Meta-learningcited by 68
  8. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  9. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 44
  10. 2022Skill-based Meta-Reinforcement Learningcited by 37
  11. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  12. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 25
  13. 2020D'ej`a vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendationcited by 14
  14. 2021Reversible Action Design for Combinatorial Optimization with Reinforcement Learningcited by 7
  15. 2022Meta Policy Learning for Cold-Start Conversational Recommendationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 1.

4 papers.

  1. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  2. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 8
  3. 2025Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systemscited by 1
  4. 2026Regret-Guided Search Control for Efficient Learning in AlphaZerocited by 0