mlsanityData to 7 Oct 2026

What Can Learned Intrinsic Rewards Capture?

arXiv 1912.05500 ↗First posted 11 Dec 2019cs.AI cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

The objective of a reinforcement learning agent is to behave so as to maximise the sum of a suitable scalar function of state: the reward. These rewards are typically given and immutable. In this paper, we instead consider the proposition that the reward function itself can be a good locus of learned knowledge. To investigate this, we propose a scalable meta-gradient framework for learning useful intrinsic reward functions across multiple lifetimes of experience. Through several proof-of-concept experiments, we show that it is feasible to learn and capture knowledge about long-term exploration and exploitation into a reward function. Furthermore, we show that unlike policy transfer methods that capture "how" the agent should behave, the learned reward functions can generalise to other kinds of agents and to changes in the dynamics of the environment by capturing "what" the agent should strive to do.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 7; 2024: 5; 2025: 2; 2026: 2.

36 papers.

Sort:

By area:

  1. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  2. 2021On the Expressivity of Markov Rewardcited by 71
  3. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  4. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  5. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  6. 2022Redeeming Intrinsic Rewards via Constrained Optimizationcited by 24
  7. 2024A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Healthcited by 24
  8. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
  9. 2023Behavior Alignment via Reward Function Optimizationcited by 16
  10. 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
  11. 2022Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width: Extended Versioncited by 12
  12. 2021A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learningcited by 11
24 more
  1. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  2. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learningcited by 11
  3. 2021Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Versioncited by 9
  4. 2022Learning Synthetic Environments and Reward Networks for Reinforcement Learningcited by 6
  5. 2023General Policies, Subgoal Structure, and Planning Widthcited by 6
  6. 2021Target Languages (vs. Inductive Biases) for Learning to Act and Plancited by 5
  7. 2021Wasserstein Distance Maximizing Intrinsic Controlcited by 5
  8. 2021Hindsight Foresight Relabeling for Meta-Reinforcement Learningcited by 4
  9. 2022An Investigation of the Bias-Variance Tradeoff in Meta-Gradientscited by 4
  10. 2021Adaptive Pairwise Weights for Temporal Credit Assignmentcited by 2
  11. 2021Neural Auto-Curriculacited by 2
  12. 2024Evolution of Rewards for Food and Motor Action by Simulating Birth and Deathcited by 2
  13. 2024Sketch Decompositions for Classical Planning via Deep Reinforcement Learningcited by 2
  14. 2023Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIscited by 1
  15. 2024Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedbackcited by 1
  16. 2024Black Box Meta-Learning Intrinsic Rewardscited by 1
  17. 2021Delayed Rewards Calibration via Reward Empirical Sufficiencycited by 0
  18. 2022Language-Based Causal Representation Learningcited by 0
  19. 2022Reusable Options through Gradient-based Meta Learningcited by 0
  20. 2023Mastering Asymmetrical Multiplayer Game with Multi-Agent Asymmetric-Evolution Reinforcement Learningcited by 0
  21. 2025Learning Task Belief Similarity with Latent Dynamics for Meta-Reinforcement Learningcited by 0
  22. 2025Subjective functionscited by 0
  23. 2026When Adaptive Rewards Hurt: Causal Probing and the Switching-Stability Dilemma in LLM-Guided LEO Satellite Schedulingcited by 0
  24. 2026Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximizationcited by 0