mlsanityData to 7 Oct 2026

Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning

arXiv 2409.20258 ↗First posted 30 Sep 2024cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Many decision-making problems feature multiple objectives where it is not always possible to know the preferences of a human or agent decision-maker for different objectives. However, demonstrated behaviors from the decision-maker are often available. This research proposes a dynamic weight-based preference inference (DWPI) algorithm that can infer the preferences of agents acting in multi-objective decision-making problems from demonstrations. The proposed algorithm is evaluated on three multi-objective Markov decision processes: Deep Sea Treasure, Traffic, and Item Gathering, and is compared to two existing preference inference algorithms. Empirical results demonstrate significant improvements compared to the baseline algorithms, in terms of both time efficiency and inference accuracy. The DWPI algorithm maintains its performance when inferring preferences for sub-optimal demonstrations. Moreover, the DWPI algorithm does not necessitate any interactions with the user during inference - only demonstrations are required. We provide a correctness proof and complexity analysis of the algorithm and statistically evaluate the performance under different representation of demonstrations.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

19 of 47 references.

References · 19

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  5. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  6. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  7. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 96
  8. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  9. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
  10. 2016Multi-Objective Deep Reinforcement Learningcited by 81
  11. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  12. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
7 more
  1. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 25
  2. 2018Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Makingcited by 17
  3. 2023Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedbackcited by 7
  4. 2023Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning: A Dynamic Weight-based Approachcited by 3
  5. 2023Preference Inference from Demonstration in Multi-objective Multi-agent Decision Makingcited by 3
  6. 2024Go-Explore for Residential Energy Managementcited by 3
  7. 2024Inferring Preferences from Demonstrations in Multi-Objective Residential Energy Managementcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Automated Generation of MDPs Using Logic Programming and LLMs for Robotic Applicationscited by 2