mlsanityData to 7 Oct 2026

Evolved Policy Gradients

arXiv 1802.04821 ↗First posted 13 Feb 2018cs.LG cs.AIcited by 86

Not on the map. Why?

Abstract (arXiv)

We propose a metalearning approach for learning gradient-based reinforcement learning (RL) algorithms. The idea is to evolve a differentiable loss function, such that an agent, which optimizes its policy to minimize this loss, will achieve high rewards. The loss is parametrized via temporal convolutions over the agent's experience. Because this loss is highly flexible in its ability to take into account the agent's history, it enables fast task learning. Empirical results show that our evolved policy gradient algorithm (EPG) achieves faster learning on several randomized environments compared to an off-the-shelf policy gradient method. We also demonstrate that EPG's learned loss can generalize to out-of-distribution test time tasks, and exhibits qualitatively different behavior from other popular metalearning algorithms.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 29; 2022: 17; 2023: 22; 2024: 12; 2025: 1; 2026: 2.

86 papers.

Sort:

By area:

  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  2. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  3. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 109
  4. 2022Discovered Policy Optimisationcited by 103
  5. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 102
  6. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  7. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  8. 2022A Survey on Model-based Reinforcement Learningcited by 65
  9. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  10. 2021Meta-learning PINN loss functionscited by 51
  11. 2021Evolving Reinforcement Learning Algorithmscited by 48
  12. 2024Discovering Preference Optimization Algorithms with and for Large Language Modelscited by 38
38 more
  1. 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
  2. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  3. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  4. 2021DACBench: A Benchmark Library for Dynamic Algorithm Configurationcited by 28
  5. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
  6. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  7. 2021Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learningcited by 25
  8. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  9. 2024Task-driven Image Fusion with Learnable Fusion Losscited by 24
  10. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  11. 2021Doubly Robust Off-Policy Actor-Critic: Convergence and Optimalitycited by 20
  12. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  13. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  14. 2024Discovering Temporally-Aware Reinforcement Learning Algorithmscited by 18
  15. 2023Fine-grained Recognition with Learnable Semantic Data Augmentationcited by 17
  16. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
  17. 2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
  18. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  19. 2021CoMPS: Continual Meta Policy Searchcited by 15
  20. 2023Unsupervised Meta-Learning via Few-shot Pseudo-supervised Contrastive Learningcited by 15
  21. 2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 15
  22. 2023ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learningcited by 15
  23. 2021Searching for Robustness: Loss Learning for Noisy Classification Taskscited by 13
  24. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  25. 2022Reinforcement Learning with Automated Auxiliary Loss Searchcited by 12
  26. 2023Learning to Optimize for Reinforcement Learningcited by 12
  27. 2023Efficient automatic design of robotscited by 11
  28. 2024Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timestepscited by 11
  29. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  30. 2022Learning Symbolic Model-Agnostic Loss Functions via Meta-Learningcited by 9
  31. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  32. 2026Harnessing Agentic Evolutioncited by 8
  33. 2021MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerancecited by 7
  34. 2023Deep Model Predictive Optimizationcited by 7
  35. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  36. 2024EvIL: Evolution Strategies for Generalisable Imitation Learningcited by 7
  37. 2020FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularizationcited by 6
  38. 2021On the Practical Consistency of Meta-Reinforcement Learning Algorithmscited by 6

50 most cited of 86 All 86 (JSON)