mlsanityData to 7 Oct 2026

A Practical Guide to Multi-Objective Reinforcement Learning and Planning

arXiv 2103.09568 ↗First posted 17 Mar 2021cs.AI cs.LGcited by 240

Not on the map. Why?

Cited by 29 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

Key facts. Cited by 240 papers in the map (data through 7 Oct 2026). Builds on 33 of its 225 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

33 of 225 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  5. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  6. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 229
  7. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 199
  8. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  9. 2018Universal Successor Features Approximatorscited by 102
  10. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 96
  11. 2019Deep Reinforcement Learning for Multi-objective Optimizationcited by 87
  12. 2016Multi-Objective Deep Reinforcement Learningcited by 81
21 more
  1. 2020Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewardscited by 60
  2. 2020A Distributional View on Multi-Objective Policy Optimizationcited by 51
  3. 2019Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Surveycited by 45
  4. 2018Meta-Learning for Multi-objective Reinforcement Learningcited by 33
  5. 2020Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learningcited by 32
  6. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  7. 2019Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learningcited by 25
  8. 2018A Multi-Objective Deep Reinforcement Learning Frameworkcited by 22
  9. 2018Multi-objective Contextual Bandit Problem with Similarity Informationcited by 19
  10. 2018Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Makingcited by 17
  11. 2018Universal Successor Representations for Transfer Reinforcement Learningcited by 10
  12. 2020A utility-based analysis of equilibria in multi-objective normal form gamescited by 9
  13. 2020Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Gamescited by 5
  14. 2021Scalable Optimization for Wind Farm Control using Coordination Graphscited by 5
  15. 2021Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Searchcited by 5
  16. 2013Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processescited by 4
  17. 2017Identification and Off-Policy Learning of Multiple Objectives Using Adaptive Clusteringcited by 4
  18. 2019Fleetwide data-enabled reliability improvement of wind turbinescited by 4
  19. 2018Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processescited by 3
  20. 2018Interpretable Multi-Objective Reinforcement Learning through Policy Orchestrationcited by 3
  21. 2019Relationship Explainable Multi-objective Reinforcement Learning with Semantic Explainability Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 21; 2023: 30; 2024: 62; 2025: 51; 2026: 72.

240 papers.

Sort:

By area:

  1. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  2. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  3. 2024A Roadmap to Pluralistic Alignmentcited by 175
  4. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  5. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
  6. 2022Multi-Objective GFlowNetscited by 100
  7. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  8. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 51
  9. 2024Value Augmented Sampling for Language Model Alignment and Personalizationcited by 49
  10. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
  11. 2024Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuningcited by 42
  12. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
38 more
  1. 2023Multi-Objective Reinforcement Learning Based on Decomposition: A Taxonomy and Frameworkcited by 28
  2. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 25
  3. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  4. 2025Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weightingcited by 23
  5. 2022Pareto Conditioned Networkscited by 22
  6. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  7. 2025MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problemscited by 21
  8. 2022Deep Reinforcement Learning for Adaptive Mesh Refinementcited by 19
  9. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 18
  10. 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
  11. 2024Foundations of Multivariate Distributional Reinforcement Learningcited by 16
  12. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 15
  13. 2022gym-DSSAT: a crop model turned into a Reinforcement Learning environmentcited by 13
  14. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 13
  15. 2025Multi-Objective Preference Optimization: Improving Human Alignment of Generative Modelscited by 13
  16. 2021Vector Optimization with Stochastic Bandit Feedbackcited by 10
  17. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 10
  18. 2026Reward-free Alignment for Conflicting Objectivescited by 10
  19. 2023Challenges for Reinforcement Learning in Quantum Circuit Designcited by 9
  20. 2024Policy Aggregationcited by 8
  21. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learningcited by 8
  22. 2026Vector Policy Optimization: Training for Diversity Improves Test-Time Searchcited by 8
  23. 2023Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedbackcited by 7
  24. 2023Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifestocited by 7
  25. 2025Preference-based Multi-Objective Reinforcement Learningcited by 7
  26. 2025PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modelingcited by 7
  27. 2021Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Makingcited by 6
  28. 2022Value Function Decomposition for Iterative Design of Reinforcement Learning Agentscited by 6
  29. 2023Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learningcited by 6
  30. 2024Adaptive Alignment: Dynamic Preference Adjustments via Multi-Objective Reinforcement Learning for Pluralistic AIcited by 6
  31. 2024Multi-objective Deep Learning: Taxonomy and Survey of the State of the Artcited by 6
  32. 2025Adaptive Helpfulness-Harmlessness Alignment with Preference Vectorscited by 6
  33. 2026On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignmentcited by 6
  34. 2022Multi-Agent Reinforcement Learning for Adaptive Mesh Refinementcited by 5
  35. 2023PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Marketscited by 5
  36. 2023Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learningcited by 5
  37. 2023What Lies beyond the Pareto Front? A Survey on Decision-Support Methods for Multi-Objective Optimizationcited by 5
  38. 2024MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learningcited by 5

50 most cited of 240 All 240 (JSON)