# On the Pros and Cons of Active Learning for Moral Preference Elicitation

arXiv: https://arxiv.org/abs/2407.18889 · HTML: https://mlsanity.com/paper/2407.18889/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 72 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Practical Obstacles to Deploying Active Learning](https://mlsanity.com/paper/1807.04801/) (2018)
- [Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering](https://mlsanity.com/paper/2107.02331/) (2021)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Adapting a Kidney Exchange Algorithm to Align with Human Values](https://mlsanity.com/paper/2005.09755/) (2020)
- [Active Learning under Label Shift](https://mlsanity.com/paper/2007.08479/) (2020)
- [Mathematical foundations of moral preferences](https://mlsanity.com/paper/2101.08193/) (2021)
- [A Causal Analysis of Harm](https://mlsanity.com/paper/2210.05327/) (2022)
- [Robust Active Preference Elicitation](https://mlsanity.com/paper/2003.01899/) (2020)
- [On the Limitations of Simulating Active Learning](https://mlsanity.com/paper/2305.13342/) (2023)
- [Indecision Modeling](https://mlsanity.com/paper/2012.08485/) (2020)
- [Deploying a Robust Active Preference Elicitation Algorithm on MTurk: Experiment Design, Interface, and Evaluation for COVID-19 Patient Prioritization](https://mlsanity.com/paper/2306.04061/) (2023)
- [Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems](https://mlsanity.com/paper/2402.16751/) (2024)
- [Limitations of Assessing Active Learning Performance at Runtime](https://mlsanity.com/paper/1901.10338/) (2019)
- [Investigating Multi-source Active Learning for Natural Language Inference](https://mlsanity.com/paper/2302.06976/) (2023)

## Built on by (most cited first, 7 of 7)

- [Can Language Models Reason about Individualistic Human Values and Preferences?](https://mlsanity.com/paper/2410.03868/) (2024)
- [Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions](https://mlsanity.com/paper/2503.00940/) (2025)
- [Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English](https://mlsanity.com/paper/2508.00238/) (2025)
- [SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing](https://mlsanity.com/paper/2603.01630/) (2026)
- [The End Justifies the Mean: A Linear Ranking Rule for Proportional Sequential Decisions](https://mlsanity.com/paper/2605.12717/) (2026)
- [Internal Pluralism and the Limits of Pairwise Comparisons](https://mlsanity.com/paper/2607.02672/) (2026)
- [Algorithmic Impact Reveals the Hidden Social Choice Structure of Alignment](https://mlsanity.com/paper/2608.24046/) (2026)

## Abstract (arXiv)

Computational preference elicitation methods are tools used to learn people's preferences quantitatively in a given context. Recent works on preference elicitation advocate for active learning as an efficient method to iteratively construct queries (framed as comparisons between context-specific cases) that are likely to be most informative about an agent's underlying preferences. In this work, we argue that the use of active learning for moral preference elicitation relies on certain assumptions about the underlying moral preferences, which can be violated in practice. Specifically, we highlight the following common assumptions (a) preferences are stable over time and not sensitive to the sequence of presented queries, (b) the appropriate hypothesis class is chosen to model moral preferences, and (c) noise in the agent's responses is limited. While these assumptions can be appropriate for preference elicitation in certain domains, prior research on moral psychology suggests they may not be valid for moral judgments. Through a synthetic simulation of preferences that violate the above assumptions, we observe that active learning can have similar or worse performance than a basic random query selection method in certain settings. Yet, simulation results also demonstrate that active learning can still be viable if the degree of instability or noise is relatively small and when the agent's preferences can be approximately represented with the hypothesis class used for learning. Our study highlights the nuances associated with effective moral preference elicitation in practice and advocates for the cautious use of active learning as a methodology to learn moral preferences.