mlsanityData to 14 Oct 2026

PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm

arXiv 2208.07914 ↗First posted 16 Aug 2022cs.LG cs.AIcited by 58

Not on the map. Why?

Cited by 33 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multi-objective reinforcement learning (MORL) approaches have emerged to tackle many real-world problems with multiple conflicting objectives by maximizing a joint objective function weighted by a preference vector. These approaches find fixed customized policies corresponding to preference vectors specified during training. However, the design constraints and objectives typically change dynamically in real-life scenarios. Furthermore, storing a policy for each potential preference is not scalable. Hence, obtaining a set of Pareto front solutions for the entire preference space in a given domain with a single training is critical. To this end, we propose a novel MORL algorithm that trains a single universal network to cover the entire preference space scalable to continuous robotic tasks. The proposed approach, Preference-Driven MORL (PD-MORL), utilizes the preferences as guidance to update the network parameters. It also employs a novel parallelization approach to increase sample efficiency. We show that PD-MORL achieves up to 25% larger hypervolume for challenging continuous control tasks and uses an order of magnitude fewer trainable parameters compared to prior approaches.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 17 of its 35 references that are in the map. First posted on arXiv 16 Aug 2022.

Builds on

17 of 35 references.

References · 17

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  6. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 242
  7. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 177
  8. 2020Learning the Pareto Front with Hypernetworkscited by 125
  9. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 100
  10. 2016Multi-Objective Deep Reinforcement Learningcited by 83
  11. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  12. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
5 more
  1. 2020A Distributional View on Multi-Objective Policy Optimizationcited by 52
  2. 2020A Review of Deep Reinforcement Learning for Smart Building Energy Managementcited by 38
  3. 2018Meta-Learning for Multi-objective Reinforcement Learningcited by 34
  4. 2014Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation Supplementary Materialcited by 14
  5. 2020Multi-objective Reinforcement Learning based approach for User-Centric Power Optimization in Smart Home Environmentscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 18; 2025: 13; 2026: 25.

58 papers.

Sort:

By area:

  1. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 94
  2. 2024Panacea: Pareto Alignment via Preference Adaptation for LLMscited by 80
  3. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 20
  4. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  5. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 15
  6. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 13
  7. 2024PMGDA: A Preference-based Multiple Gradient Descent Algorithmcited by 10
  8. 2025Robust Multi-Objective Controlled Decoding of Large Language Modelscited by 9
  9. 2024Partial Rankings of Optimizerscited by 8
  10. 2024Learning Pareto Set for Multi-Objective Continuous Robot Controlcited by 8
  11. 2025OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignmentcited by 7
  12. 2023IBCL: Zero-shot Model Generation under Stability-Plasticity Trade-offscited by 5
38 more
  1. 2024Conflict-Averse Gradient Aggregation for Constrained Multi-Objective Reinforcement Learningcited by 5
  2. 2024An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learningcited by 5
  3. 2024How to Find the Exact Pareto Front for Multi-Objective MDPs?cited by 5
  4. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  5. 2024C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Frontcited by 4
  6. 2024Scalable Multi-Objective Reinforcement Learning with Fairness Guarantees using Lorenz Dominancecited by 4
  7. 2024Federated Communication-Efficient Multi-Objective Optimizationcited by 3
  8. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Drivingcited by 3
  9. 2026Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimizationcited by 3
  10. 2026Property-driven Protein Inverse Folding With Multi-Objective Preference Alignmentcited by 3
  11. 2024Demonstration Guided Multi-Objective Reinforcement Learningcited by 2
  12. 2024Demonstration-Enhanced Adaptable Multi-Objective Robot Navigationcited by 2
  13. 2024A Meta-Learning Approach for Multi-Objective Reinforcement Learning in Sustainable Home Environmentscited by 2
  14. 2024Preference-Optimized Pareto Set Learning for Blackbox Optimizationcited by 2
  15. 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learningcited by 2
  16. 2025FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learningcited by 2
  17. 2026GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learningcited by 2
  18. 2026From Intents to Actions: Agentic AI in Autonomous Networkscited by 2
  19. 2026Uncovering Cross-Objective Interference in Multi-Objective Alignmentcited by 2
  20. 2026MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Roboticscited by 2
  21. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learningcited by 2
  22. 2023Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learningcited by 1
  23. 2025Reward Dimension Reduction for Scalable Multi-Objective Reinforcement Learningcited by 1
  24. 2026Learning Policy Representations for Steerable Behavior Synthesiscited by 1
  25. 2026Learning What Matters Now: Dynamic Preference Inference under Contextual Shiftscited by 1
  26. 2026A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Setscited by 1
  27. 2026Controllability in preference-conditioned multi-objective reinforcement learningcited by 1
  28. 2026ORACLE: A Multi-Objective Reinforcement Learning-Based Analog Circuit Design Optimizer with Large Language Models-Guided Explorationcited by 1
  29. 2024UCB-driven Utility Function Search for Multi-objective Reinforcement Learningcited by 0
  30. 2025Learning Pareto-Optimal Rewards from Noisy Preferences: A Framework for Multi-Objective Inverse Reinforcement Learningcited by 0
  31. 2025Interpretability by Design for Efficient Multi-Objective Reinforcement Learningcited by 0
  32. 2025Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Controlcited by 0
  33. 2025Limitations of Scalarisation in MORL: A Comparative Study in Discrete Environmentscited by 0
  34. 2026Post Hoc Extraction of Pareto Fronts for Continuous Controlcited by 0
  35. 2026PCHC: Enabling Preference Conditioned Humanoid Control via Multi-Objective Reinforcement Learningcited by 0
  36. 2026Constrained Multi-Objective Reinforcement Learning with Max-Min Criterioncited by 0
  37. 2026TetraRL: A Self-Adaptive Runtime for On-Device Deep Reinforcement Learning Systemscited by 0
  38. 2026Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learningcited by 0

50 most cited of 58 All 58 (JSON)