# Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making

arXiv: https://arxiv.org/abs/1802.07606 · HTML: https://mlsanity.com/paper/1802.07606/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2018.

## Built on by (most cited first, 18 of 18)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems](https://mlsanity.com/paper/2402.16751/) (2024)
- [Exact Pareto Optimal Search for Multi-Task Learning and Multi-Criteria Decision-Making](https://mlsanity.com/paper/2108.00597/) (2021)
- [What Lies beyond the Pareto Front? A Survey on Decision-Support Methods for Multi-Objective Optimization](https://mlsanity.com/paper/2311.11288/) (2023)
- [Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2401.02160/) (2024)
- [Interactive Groupwise Comparison for Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2507.04340/) (2025)
- [Preference Communication in Multi-Objective Normal-Form Games](https://mlsanity.com/paper/2111.09191/) (2021)
- [Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning: A Dynamic Weight-based Approach](https://mlsanity.com/paper/2304.14115/) (2023)
- [Multi-objective optimisation via the R2 utilities](https://mlsanity.com/paper/2305.11774/) (2023)
- [Inferring Preferences from Demonstrations in Multi-Objective Residential Energy Management](https://mlsanity.com/paper/2401.07722/) (2024)
- [Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2409.20258/) (2024)
- [Reward Dimension Reduction for Scalable Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2502.20957/) (2025)
- [Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation](https://mlsanity.com/paper/2506.03062/) (2025)
- [Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit](https://mlsanity.com/paper/2311.14003/) (2023)
- [DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization](https://mlsanity.com/paper/2512.14263/) (2025)
- [Preference Guided Iterated Pareto Referent Optimisation for Accessible Route Planning](https://mlsanity.com/paper/2604.00795/) (2026)
- [Efficient Personalization of Generative User Interfaces](https://mlsanity.com/paper/2604.09876/) (2026)

## Abstract (arXiv)

In multi-objective decision planning and learning, much attention is paid to producing optimal solution sets that contain an optimal policy for every possible user preference profile. We argue that the step that follows, i.e, determining which policy to execute by maximising the user's intrinsic utility function over this (possibly infinite) set, is under-studied. This paper aims to fill this gap. We build on previous work on Gaussian processes and pairwise comparisons for preference modelling, extend it to the multi-objective decision support scenario, and propose new ordered preference elicitation strategies based on ranking and clustering. Our main contribution is an in-depth evaluation of these strategies using computer and human-based experiments. We show that our proposed elicitation strategies outperform the currently used pairwise methods, and found that users prefer ranking most. Our experiments further show that utilising monotonicity information in GPs by using a linear prior mean at the start and virtual comparisons to the nadir and ideal points, increases performance. We demonstrate our decision support framework in a real-world study on traffic regulation, conducted with the city of Amsterdam.