mlsanityData to 7 Oct 2026

A Survey of Multi-Objective Sequential Decision-Making

arXiv 1402.0590 ↗First posted 4 Feb 2014cs.AIcited by 229

Not on the map. Why?

Cited by 29 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.

Key facts. Cited by 229 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 18; 2022: 31; 2023: 29; 2024: 42; 2025: 46; 2026: 62.

229 papers.

Sort:

By area:

  1. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  2. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  3. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  4. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  5. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  6. 2022Multi-Objective GFlowNetscited by 100
  7. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  8. 2023ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planningcited by 77
  9. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  10. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 51
  11. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
  12. 2024Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuningcited by 42
38 more
  1. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generationcited by 35
  2. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 28
  3. 2023Multi-Objective Reinforcement Learning Based on Decomposition: A Taxonomy and Frameworkcited by 28
  4. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  5. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  6. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  7. 2025Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weightingcited by 23
  8. 2021Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPscited by 22
  9. 2022Pareto Conditioned Networkscited by 22
  10. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  11. 2025MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problemscited by 21
  12. 2021Concave Utility Reinforcement Learning with Zero-Constraint Violationscited by 18
  13. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 18
  14. 2022Lexicographic Multi-Objective Reinforcement Learningcited by 17
  15. 2024Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspectivecited by 16
  16. 2024Foundations of Multivariate Distributional Reinforcement Learningcited by 16
  17. 2022On the link between conscious function and general intelligence in humans and machinescited by 15
  18. 2022Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learningcited by 15
  19. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 15
  20. 2022Socially Fair Reinforcement Learningcited by 13
  21. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 13
  22. 2025Multi-Objective Preference Optimization: Improving Human Alignment of Generative Modelscited by 13
  23. 2022Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestrationcited by 12
  24. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 10
  25. 2026Reward-free Alignment for Conflicting Objectivescited by 10
  26. 2021Subdimensional Expansion for Multi-objective Multi-agent Path Findingcited by 9
  27. 2022Exploring the Pareto front of multi-objective COVID-19 mitigation policies using reinforcement learningcited by 9
  28. 2024On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Taskscited by 9
  29. 2024Multi-objective Reinforcement learning from AI Feedbackcited by 9
  30. 2024Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewardscited by 9
  31. 2026Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRcited by 9
  32. 2021Factored Policy Gradients: Leveraging Structure for Efficient Learning in MOMDPscited by 8
  33. 2023Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learningcited by 8
  34. 2023Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewardscited by 8
  35. 2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policiescited by 8
  36. 2024Policy Aggregationcited by 8
  37. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learningcited by 8
  38. 2026Vector Policy Optimization: Training for Diversity Improves Test-Time Searchcited by 8

50 most cited of 229 All 229 (JSON)