mlsanityData to 7 Oct 2026

A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation

arXiv 1908.08342 ↗First posted 21 Aug 2019cs.LG cs.AIcited by 166

Not on the map. Why?

Abstract (arXiv)

We introduce a new algorithm for multi-objective reinforcement learning (MORL) with linear preferences, with the goal of enabling few-shot adaptation to new tasks. In MORL, the aim is to learn policies over multiple competing objectives whose relative importance (preferences) is unknown to the agent. While this alleviates dependence on scalar reward design, the expected return of a policy can change significantly with varying preferences, making it challenging to learn a single model to produce optimal policies under different preference conditions. We propose a generalized version of the Bellman equation to learn a single parametric representation for optimal policies over the space of all possible preferences. After an initial learning phase, our agent can execute the optimal policy under any given preference, or automatically infer an underlying preference with very few samples. Experiments across four different domains demonstrate the effectiveness of our approach.

Key facts. Cited by 166 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 24; 2023: 18; 2024: 37; 2025: 31; 2026: 42.

166 papers.

Sort:

By area:

  1. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  2. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  3. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  4. 2024A Roadmap to Pluralistic Alignmentcited by 175
  5. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  6. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  7. 2022Multi-Objective GFlowNetscited by 100
  8. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 81
  9. 2024Panacea: Pareto Alignment via Preference Adaptation for LLMscited by 78
  10. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  11. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 51
  12. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
38 more
  1. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
  2. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  3. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
  4. 2023Multi-Objective Reinforcement Learning Based on Decomposition: A Taxonomy and Frameworkcited by 28
  5. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 25
  6. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  7. 2022Redeeming Intrinsic Rewards via Constrained Optimizationcited by 24
  8. 2021Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPscited by 22
  9. 2022Pareto Conditioned Networkscited by 22
  10. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  11. 2022Welfare and Fairness in Multi-objective Reinforcement Learningcited by 20
  12. 2025Pareto Multi-Objective Alignment for Language Modelscited by 20
  13. 2022Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systemscited by 18
  14. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 18
  15. 2025GARDO: Reinforcing Diffusion Models without Reward Hackingcited by 17
  16. 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
  17. 2022Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learningcited by 15
  18. 2022Socially Fair Reinforcement Learningcited by 13
  19. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 13
  20. 2024Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learningcited by 13
  21. 2023Continuation Path Learning for Homotopy Optimizationcited by 12
  22. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 10
  23. 2021Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learningscited by 9
  24. 2022Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learningcited by 9
  25. 2024Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewardscited by 9
  26. 2023Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learningcited by 8
  27. 2023Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directionscited by 8
  28. 2024A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimizationcited by 8
  29. 2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policiescited by 8
  30. 2024Policy Aggregationcited by 8
  31. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learningcited by 8
  32. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 8
  33. 2026Vector Policy Optimization: Training for Diversity Improves Test-Time Searchcited by 8
  34. 2023Multi-objective Reinforcement Learning with Nonlinear Preferences: Provable Approximation for Maximizing Expected Scalarized Returncited by 7
  35. 2024Thresholded Lexicographic Ordered Multiobjective Reinforcement Learningcited by 7
  36. 2025Preference-based Multi-Objective Reinforcement Learningcited by 7
  37. 2021Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Makingcited by 6
  38. 2022Design of Detectors at the Electron Ion Collider with Artificial Intelligencecited by 6

50 most cited of 166 All 166 (JSON)