# Information efficient learning of complexly structured preferences: Elicitation procedures and their application to decision making under uncertainty

arXiv: https://arxiv.org/abs/2110.12879 · HTML: https://mlsanity.com/paper/2110.12879/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 71 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 5 of 5)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Hybrid Approach to Reasoning with Partially Elicited Preference Models](https://mlsanity.com/paper/1301.6702/) (2013)
- [Efficiently Discovering Locally Exceptional yet Globally Representative Subgroups](https://mlsanity.com/paper/1709.07941/) (2017)
- [Discovering Reliable Dependencies from Data: Hardness and Improved Algorithms](https://mlsanity.com/paper/1809.05467/) (2018)
- [Robust decision analysis under severe uncertainty and ambiguous tradeoffs: an invasive species case study](https://mlsanity.com/paper/2103.04721/) (2021)

## Built on by (most cited first, 10 of 10)

- [Statistical Comparisons of Classifiers by Generalized Stochastic Dominance](https://mlsanity.com/paper/2209.01857/) (2022)
- [Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurement](https://mlsanity.com/paper/2306.12803/) (2023)
- [Statistical Multicriteria Benchmarking via the GSD-Front](https://mlsanity.com/paper/2406.03924/) (2024)
- [Statistical Multicriteria Evaluation of LLM-Generated Text](https://mlsanity.com/paper/2506.18082/) (2025)
- [Multi-Target Decision Making under Conditions of Severe Uncertainty](https://mlsanity.com/paper/2212.06832/) (2022)
- [A Statistical Case Against Empirical Human-AI Alignment](https://mlsanity.com/paper/2502.14581/) (2025)
- [Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms](https://mlsanity.com/paper/2304.09872/) (2023)
- [In all LikelihoodS: How to Reliably Select Pseudo-Labeled Data for Self-Training in Semi-Supervised Learning](https://mlsanity.com/paper/2303.01117/) (2023)
- [Reciprocal Learning](https://mlsanity.com/paper/2408.06257/) (2024)
- [Empirical Decision Theory](https://mlsanity.com/paper/2512.05677/) (2025)

## Abstract (arXiv)

In this paper we propose efficient methods for elicitation of complexly structured preferences and utilize these in problems of decision making under (severe) uncertainty. Based on the general framework introduced in Jansen, Schollmeyer and Augustin (2018, Int. J. Approx. Reason), we now design elicitation procedures and algorithms that enable decision makers to reveal their underlying preference system (i.e. two relations, one encoding the ordinal, the other the cardinal part of the preferences) while having to answer as few as possible simple ranking questions. Here, two different approaches are followed. The first approach directly utilizes the collected ranking data for obtaining the ordinal part of the preferences, while their cardinal part is constructed implicitly by measuring meta data on the decision maker's consideration times. In contrast, the second approach explicitly elicits also the cardinal part of the decision maker's preference system, however, only an approximate version of it. This approximation is obtained by additionally collecting labels of preference strength during the elicitation procedure. For both approaches, we give conditions under which they produce the decision maker's true preference system and investigate how their efficiency can be improved. For the latter purpose, besides data-free approaches, we also discuss ways for effectively guiding the elicitation procedure if data from previous elicitation rounds is available. Finally, we demonstrate how the proposed elicitation methods can be utilized in problems of decision under (severe) uncertainty. Precisely, we show that under certain conditions optimal decisions can be found without fully specifying the preference system.