mlsanityData to 14 Oct 2026

It's a matter of timescale: non-linear utility in successor features and multi-objective planning and learning

arXiv 2608.25723 ↗First posted 26 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Time is of the essence when dealing with multiple reward signals and non-linear utility. In this paper we argue that the current main approaches in multi-objective RL (SER and ESR), and successor features, are insufficient. While each approach deals with non-linear effects on user utility on different timescales, none of them take into account that different effects happening on different timescales can happen within the same decision problem. We motivate that this can indeed be the case by an example, both intuitively and numerically, leading to a new perspective, and a significant and non-trivial gap in the literature.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

20 of 39 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  4. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  5. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  6. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 242
  7. 2021Learning One Representation to Optimize All Rewardscited by 115
  8. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 100
  9. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  10. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  11. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  12. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
8 more
  1. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  2. 2024A Distributional Analogue to the Successor Representationcited by 17
  3. 2024Distributional Successor Features Enable Zero-Shot Policy Optimizationcited by 8
  4. 2023On The Expressivity of Objective-Specification Formalisms in Reinforcement Learningcited by 5
  5. 2024Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learningcited by 4
  6. 2026Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilitiescited by 3
  7. 2023Distributional Multi-Objective Decision Makingcited by 2
  8. 2025Constructing an Optimal Behavior Basis for the Option Keyboardcited by 2

Built on by

Not cited yet.