mlsanityData to 7 Oct 2026

Meta-Gradient Reinforcement Learning

arXiv 1805.09801 ↗First posted 24 May 2018cs.LG cs.AI stat.MLcited by 156

Not on the map. Why?

Abstract (arXiv)

The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. This proxy is typically based on a sampled and bootstrapped approximation to the true value function, known as a return. The particular choice of return is one of the chief components determining the nature of the algorithm: the rate at which future rewards are discounted; when and how values should be bootstrapped; or even the nature of the rewards themselves. It is well-known that these decisions are crucial to the overall success of RL algorithms. We discuss a gradient-based meta-learning algorithm that is able to adapt the nature of the return, online, whilst interacting and learning from the environment. When applied to 57 games on the Atari 2600 environment over 200 million frames, our algorithm achieved a new state-of-the-art performance.

Key facts. Cited by 156 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 48; 2022: 33; 2023: 22; 2024: 18; 2025: 18; 2026: 16.

156 papers.

Sort:

By area:

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  3. 2021ETA Prediction with Graph Neural Networks in Google Mapscited by 130
  4. 2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
  5. 2022Discovered Policy Optimisationcited by 103
  6. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 102
  7. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  8. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  9. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  10. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  11. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  12. 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
38 more
  1. 2021Bootstrapped Meta-Learningcited by 45
  2. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  3. 2021Multi-Objective Meta Learningcited by 39
  4. 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
  5. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  6. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  7. 2023Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimizationcited by 33
  8. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  9. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  10. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 22
  11. 2023Causal Reinforcement Learning: A Surveycited by 22
  12. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  13. 2022Meta Knowledge Distillationcited by 20
  14. 2022Bayesian Generational Population-Based Trainingcited by 20
  15. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  16. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  17. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  18. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
  19. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  20. 2024Discovering Temporally-Aware Reinforcement Learning Algorithmscited by 18
  21. 2022TorchOpt: An Efficient Library for Differentiable Optimizationcited by 17
  22. 2023Context Shift Reduction for Offline Meta-Reinforcement Learningcited by 17
  23. 2022On the Role of Discount Factor in Offline Reinforcement Learningcited by 16
  24. 2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
  25. 2023Behavior Alignment via Reward Function Optimizationcited by 16
  26. 2024In-context Exploration-Exploitation for Reinforcement Learningcited by 16
  27. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  28. 2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 15
  29. 2024SymbolicAI: A framework for logic-based approaches combining generative models and solverscited by 15
  30. 2021Training Learned Optimizers with Randomly Initialized Learned Optimizerscited by 13
  31. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  32. 2025DataRater: Meta-Learned Dataset Curationcited by 13
  33. 2022Continuous-Time Meta-Learning with Forward Mode Differentiationcited by 12
  34. 2021A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learningcited by 11
  35. 2022Meta-Gradients in Non-Stationary Environmentscited by 11
  36. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  37. 2022Automated Reinforcement Learning: An Overviewcited by 9
  38. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8

50 most cited of 156 All 156 (JSON)