mlsanityData to 14 Oct 2026

Guided Meta-Policy Search

arXiv 1904.00956 ↗First posted 1 Apr 2019cs.LG cs.AI cs.ROcited by 26

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) algorithms have demonstrated promising results on complex tasks, yet often require impractical numbers of samples since they learn from scratch. Meta-RL aims to address this challenge by leveraging experience from previous tasks so as to more quickly solve new tasks. However, in practice, these algorithms generally also require large amounts of on-policy experience during the meta-training process, making them impractical for use in many problems. To this end, we propose to learn a reinforcement learning procedure in a federated way, where individual off-policy learners can solve the individual meta-training tasks, and then consolidate these solutions into a single meta-learner. Since the central meta-learner learns by imitating the solutions to the individual tasks, it can accommodate either the standard meta-RL problem setting or a hybrid setting where some or all tasks are provided with example demonstrations. The former results in an approach that can leverage policies learned for previous tasks without significant amounts of on-policy data during meta-training, whereas the latter is particularly useful in cases where demonstrations are easy for a person to provide. Across a number of continuous control meta-RL problems, we demonstrate significant improvements in meta-RL sample efficiency in comparison to prior work as well as the ability to scale to domains with visual observations.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 10; 2022: 5; 2023: 4; 2024: 4; 2025: 2.

26 papers.

Sort:

By area:

  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  2. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  3. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
  4. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
  5. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
  6. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 36
  7. 2021When Is Generalizable Reinforcement Learning Tractable?cited by 20
  8. 2021CoMPS: Continual Meta Policy Searchcited by 15
  9. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 14
  10. 2022MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimizationcited by 10
  11. 2023IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Makingcited by 6
  12. 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imaginationcited by 6
14 more
  1. 2021A Meta-Reinforcement Learning Approach to Process Controlcited by 5
  2. 2023Continual Robot Learning using Self-Supervised Task Inferencecited by 5
  3. 2021Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularizationcited by 4
  4. 2025Learning Policy Committees for Effective Personalization in MDPs with Diverse Taskscited by 4
  5. 2022Continual Meta-Reinforcement Learning for UAV-Aided Vehicular Wireless Networkscited by 3
  6. 2024Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneitycited by 2
  7. 2024Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparatorcited by 2
  8. 2021Off-Policy Meta-Reinforcement Learning Based on Feature Embedding Spacescited by 1
  9. 2022Enhanced Meta Reinforcement Learning using Demonstrations in Sparse Reward Environmentscited by 1
  10. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
  11. 2022Model Based Meta Learning of Critics for Policy Gradientscited by 0
  12. 2022Implicit Training of Energy Model for Structure Predictioncited by 0
  13. 2024Smart Sampling: Self-Attention and Bootstrapping for Improved Ensembled Q-Learningcited by 0
  14. 2025Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Datacited by 0