mlsanityData to 14 Oct 2026

Multi-objective Reinforcement Learning with Nonlinear Preferences: Provable Approximation for Maximizing Expected Scalarized Return

arXiv 2311.02544 ↗First posted 5 Nov 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

We study multi-objective reinforcement learning with nonlinear preferences over trajectories. That is, we maximize the expected value of a nonlinear function over accumulated rewards (expected scalarized return or ESR) in a multi-objective Markov Decision Process (MOMDP). We derive an extended form of Bellman optimality for nonlinear optimization that explicitly considers time and current accumulated reward. Using this formulation, we describe an approximation algorithm for computing an approximately optimal non-stationary policy in pseudopolynomial time for smooth scalarization functions with a constant number of rewards. We prove the approximation analytically and demonstrate the algorithm experimentally, showing that there can be a substantial gap between the optimal policy computed by our algorithm and alternative baselines.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 12 of its 33 references that are in the map. First posted on arXiv 5 Nov 2023.

Builds on

12 of 33 references.

References · 12

  1. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  2. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  4. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 177
  5. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 100
  6. 2016Fairness in Reinforcement Learningcited by 95
  7. 2020Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewardscited by 60
  8. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 48
  9. 2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
  10. 2022Welfare and Fairness in Multi-objective Reinforcement Learningcited by 20
  11. 2022Fairness and Welfare Quantification for Regret in Multi-Armed Banditscited by 17
  12. 2022Reinforcement Learning with Stepwise Fairness Constraintscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimizationcited by 3
  2. 2025FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learningcited by 2
  3. 2026Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learningcited by 2
  4. 2026A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Setscited by 1
  5. 2026Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deploymentcited by 0
  6. 2026Constrained Multi-Objective Reinforcement Learning with Max-Min Criterioncited by 0
  7. 2026AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RLcited by 0