mlsanityData to 14 Oct 2026

Controllability in preference-conditioned multi-objective reinforcement learning

arXiv 2605.10585 ↗First posted 11 May 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Multi-objective reinforcement learning (MORL) allows a user to express preference over outcomes in terms of the relative importance of the objectives, but standard metrics cannot capture whether changes in preference reliably change the agent's behavior in the intended way, a property termed controllability. As a result, preference-conditioned agents can score well on standard MORL metrics while being insensitive to the preference input. If the ability to control agents cannot be reliably assessed, the symbolic interface that MORL provides between user intent and agent behavior is broken. Mainstream MORL metrics alone fail to measure the controllability of preference-conditioned agents, motivating a complementary metric specifically designed to that end. We hope the results spur discussion in the community on existing evaluation protocols to consolidate advances in preference adaptation in MORL to larger and more complex problems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 39 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

21 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning28 Jan
  2. 2024In Search for Architectures and Loss Functions in Multi-Objective Reinforcement Learning23 Jul

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  4. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  5. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  6. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 100
  7. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
  8. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 58
  9. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 54
  10. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 48
  11. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
  12. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 19
7 more
  1. 2025AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learningcited by 9
  2. 2024MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learningcited by 6
  3. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learningcited by 6
  4. 2025The Formalism-Implementation Gap in Reinforcement Learning Researchcited by 3
  5. 2021Posetal Games: Efficiency, Existence, and Refinement of Equilibria in Games with Prioritized Metricscited by 1
  6. 2024You Can't Always Get What You Want: Games of Ordered Preferencecited by 1
  7. 2025Approximate solutions to games of ordered preferencecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PROMO: Preference-conditioned Multi-Objective Reinforcement Learning for Quadrupedal Robotscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Preference-based Multi-Objective Reinforcement Learning
  2. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RL
  3. 2024C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front
  4. 2026Objective-Behavior Alignment: Diagnostics for MORL Policy Selection
  5. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning
  6. 2026Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deployment
  7. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learning
  8. 2026On Preference Coverage Collapse from Hindsight Relabeling in Multi-Objective Reinforcement Learning