# Subjective functions

arXiv: https://arxiv.org/abs/2512.15948 · HTML: https://mlsanity.com/paper/2512.15948/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 77 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Builds on (most cited first, 14 of 14)

- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/1805.09801/) (2018)
- [Goal-Conditioned Reinforcement Learning: Problems and Solutions](https://mlsanity.com/paper/2201.08299/) (2022)
- [On Learning Intrinsic Rewards for Policy Gradient Methods](https://mlsanity.com/paper/1804.06459/) (2018)
- [Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey](https://mlsanity.com/paper/2012.09830/) (2020)
- [CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning](https://mlsanity.com/paper/1810.06284/) (2018)
- [Improving Generalization in Meta Reinforcement Learning using Learned Objectives](https://mlsanity.com/paper/1910.04098/) (2019)
- [Meta-Learning via Learned Loss](https://mlsanity.com/paper/1906.05374/) (2019)
- [Meta-Gradient Reinforcement Learning with an Objective Discovered Online](https://mlsanity.com/paper/2007.08433/) (2020)
- [What Can Learned Intrinsic Rewards Capture?](https://mlsanity.com/paper/1912.05500/) (2019)
- [Temporal Difference Uncertainties as a Signal for Exploration](https://mlsanity.com/paper/2010.02255/) (2020)
- [Characterizing the Gap Between Actor-Critic and Policy Gradient](https://mlsanity.com/paper/2106.06932/) (2021)
- [Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization](https://mlsanity.com/paper/2506.13345/) (2025)

## Abstract (arXiv)

Where do objective functions come from? How do we select what goals to pursue? Human intelligence is adept at synthesizing new objective functions on the fly. How does this work, and can we endow artificial systems with the same ability? This paper proposes an approach to answering these questions, starting with the concept of a subjective function, a higher-order objective function that is endogenous to the agent (i.e., defined with respect to the agent's features, rather than an external task). Expected prediction error is studied as a concrete example of a subjective function. This proposal has many connections to ideas in psychology, neuroscience, and machine learning.