mlsanityData to 14 Oct 2026

ProMP: Proximal Meta-Policy Search

arXiv 1810.06784 ↗First posted 16 Oct 2018cs.LG stat.MLcited by 91

Not on the map. Why?

Abstract (arXiv)

Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit assignment to pre-adaptation behavior or implement it naively. This leads to poor sample-efficiency during meta-training as well as ineffective task identification strategies. This paper provides a theoretical analysis of credit assignment in gradient-based Meta-RL. Building on the gained insights we develop a novel meta-learning algorithm that overcomes both the issue of poor credit assignment and previous difficulties in estimating meta-policy gradients. By controlling the statistical distance of both pre-adaptation and adapted policies during meta-policy search, the proposed algorithm endows efficient and stable meta-learning. Our approach leads to superior pre-adaptation policy behavior and consistently outperforms previous Meta-RL algorithms in sample-efficiency, wall-clock time, and asymptotic performance.

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 28; 2022: 28; 2023: 13; 2024: 10; 2025: 6; 2026: 4.

91 papers.

Sort:

By area:

  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  2. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  3. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  4. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 119
  5. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
  6. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
  7. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  8. 2022A Survey on Model-based Reinforcement Learningcited by 66
  9. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 63
  10. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 53
  11. 2022Transformers are Meta-Reinforcement Learnerscited by 41
  12. 2022Skill-based Meta-Reinforcement Learningcited by 37
38 more
  1. 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 37
  2. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35
  3. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  4. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 19
  5. 2023Prompt-Tuning Decision Transformer with Preference Rankingcited by 19
  6. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQRcited by 17
  7. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  8. 2021CoMPS: Continual Meta Policy Searchcited by 15
  9. 2022Generalizing Gaussian Smoothing for Random Searchcited by 15
  10. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 15
  11. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformercited by 15
  12. 2022Meta-Learning Priors for Safe Bayesian Optimizationcited by 14
  13. 2022Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?cited by 13
  14. 2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
  15. 2021A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learningcited by 12
  16. 2022Continuous-Time Meta-Learning with Forward Mode Differentiationcited by 12
  17. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  18. 2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
  19. 2022Understanding Benign Overfitting in Gradient-Based Meta Learningcited by 11
  20. 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
  21. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  22. 2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 10
  23. 2021Dynamic population-based meta-learning for multi-agent communication with natural languagecited by 10
  24. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  25. 2023Efficient Symbolic Policy Learning with Differentiable Symbolic Expressioncited by 10
  26. 2024Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Ratecited by 10
  27. 2024Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learningcited by 10
  28. 2021Linear Representation Meta-Reinforcement Learning for Instant Adaptationcited by 9
  29. 2021Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learningcited by 9
  30. 2021MAML is a Noisy Contrastive Learner in Classificationcited by 9
  31. 2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
  32. 2022Hierarchically Structured Task-Agnostic Continual Learningcited by 9
  33. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  34. 2021Meta-Reinforcement Learning by Tracking Task Non-stationaritycited by 8
  35. 2021Mixture-of-Variational-Experts for Continual Learningcited by 8
  36. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  37. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 8
  38. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimationcited by 8

50 most cited of 91 All 91 (JSON)