mlsanityData to 7 Oct 2026

A Distributional View on Multi-Objective Policy Optimization

arXiv 2005.07513 ↗First posted 15 May 2020cs.LG cs.AI cs.ROcited by 51

Not on the map. Why?

Abstract (arXiv)

Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for practitioners to express numerical preferences over objectives in their native units. In this paper we propose a novel algorithm for multi-objective reinforcement learning that enables setting desired preferences for objectives in a scale-invariant way. We propose to learn an action distribution for each objective, and we use supervised learning to fit a parametric policy to a combination of these distributions. We demonstrate the effectiveness of our approach on challenging high-dimensional real and simulated robotics tasks, and show that setting different preferences in our framework allows us to trace out the space of nondominated solutions.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 7; 2023: 14; 2024: 9; 2025: 6; 2026: 5.

51 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  3. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  4. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  5. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  6. 2022Multi-Objective GFlowNetscited by 100
  7. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  8. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  9. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 51
  10. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  11. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
  12. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
38 more
  1. 2023Boosting Continuous Control with Consistency Policycited by 36
  2. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 27
  3. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  4. 2021Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPscited by 22
  5. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  6. 2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16
  7. 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
  8. 2025Physics-Based Motion Imitation with Adversarial Differential Discriminatorscited by 11
  9. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 10
  10. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  11. 2024Policy-regularized Offline Multi-objective Reinforcement Learningcited by 9
  12. 2024Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learningcited by 9
  13. 2024A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimizationcited by 8
  14. 2021Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Searchcited by 5
  15. 2024How to Find the Exact Pareto Front for Multi-Objective MDPs?cited by 5
  16. 2021On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuningcited by 4
  17. 2021Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Plannercited by 4
  18. 2022Monte Carlo Tree Search Algorithms for Risk-Aware and Multi-Objective Reinforcement Learningcited by 4
  19. 2024Safety Optimized Reinforcement Learning via Multi-Objective Policy Optimizationcited by 4
  20. 2024Conflict-Averse Gradient Aggregation for Constrained Multi-Objective Reinforcement Learningcited by 4
  21. 2024C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Frontcited by 4
  22. 2024In Search for Architectures and Loss Functions in Multi-Objective Reinforcement Learningcited by 3
  23. 2026Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imaginationcited by 3
  24. 2025Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for Frozen LLMscited by 2
  25. 2026A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Controlcited by 2
  26. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learningcited by 2
  27. 2026Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learningcited by 2
  28. 2021Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific researchcited by 1
  29. 2022Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Modelscited by 1
  30. 2023Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learningcited by 1
  31. 2021A Quadratic Actor Network for Model-Free Reinforcement Learningcited by 0
  32. 2023A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysiscited by 0
  33. 2023A Comparative Visual Analytics Framework for Evaluating Evolutionary Processes in Multi-objective Optimizationcited by 0
  34. 2023Policy composition in reinforcement learning via multi-objective policy optimizationcited by 0
  35. 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Modelscited by 0
  36. 2025Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Lightcited by 0
  37. 2025Spatial Computing Communications for Multi-User Virtual Reality in Distributed Mobile Edge Computing Networkcited by 0
  38. 2025Iterative Foundation Model Fine-Tuning on Multiple Rewardscited by 0

50 most cited of 51 All 51 (JSON)