# Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems

arXiv: https://arxiv.org/abs/2402.16751 · HTML: https://mlsanity.com/paper/2402.16751/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 89 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [RobBERT: a Dutch RoBERTa-based Language Model](https://mlsanity.com/paper/2001.06286/) (2020)
- [Research Priorities for Robust and Beneficial Artificial Intelligence](https://mlsanity.com/paper/1602.03506/) (2016)
- [Occam's razor is insufficient to infer the preferences of irrational agents](https://mlsanity.com/paper/1712.05812/) (2017)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [ValueNet: A New Dataset for Human Value Driven Dialogue System](https://mlsanity.com/paper/2112.06346/) (2021)
- [MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction](https://mlsanity.com/paper/1904.08314/) (2019)
- [Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)](https://mlsanity.com/paper/1901.00064/) (2018)
- [The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments](https://mlsanity.com/paper/2203.14563/) (2022)
- [Responsible Autonomy](https://mlsanity.com/paper/1706.02513/) (2017)
- [Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making](https://mlsanity.com/paper/1802.07606/) (2018)
- [Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive](https://mlsanity.com/paper/2301.12534/) (2023)
- [Do Differences in Values Influence Disagreements in Online Discussions?](https://mlsanity.com/paper/2310.15757/) (2023)
- [Taking Principles Seriously: A Hybrid Approach to Value Alignment](https://mlsanity.com/paper/2012.11705/) (2020)
- [Reason Against the Machine: Future Directions for Mass Online Deliberation](https://mlsanity.com/paper/2107.12711/) (2021)
- [Enabling Value Sensitive AI Systems through Participatory Design Fictions](https://mlsanity.com/paper/1912.07381/) (2019)
- [Learning to Adapt Domain Shifts of Moral Values via Instance Weighting](https://mlsanity.com/paper/2204.07603/) (2022)
- [Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning](https://mlsanity.com/paper/2401.17228/) (2024)
- [Negative Human Rights as a Basis for Long-term AI Safety and Regulation](https://mlsanity.com/paper/2208.14788/) (2022)

## Built on by (most cited first, 7 of 7)

- [On the Pros and Cons of Active Learning for Moral Preference Elicitation](https://mlsanity.com/paper/2407.18889/) (2024)
- [On The Stability of Moral Preferences: A Problem with Computational Elicitation Methods](https://mlsanity.com/paper/2408.02862/) (2024)
- [Towards Cognitively-Faithful Decision-Making Models to Improve AI Alignment](https://mlsanity.com/paper/2509.04445/) (2025)
- [Will Annotators Disagree? Identifying Subjectivity in Value-Laden Arguments](https://mlsanity.com/paper/2509.06704/) (2025)
- [Facilitating Opinion Diversity through Hybrid NLP Approaches](https://mlsanity.com/paper/2405.09439/) (2024)
- [Taking a SEAT: Predicting Value Interpretations from Sentiment, Emotion, Argument, and Topic Annotations](https://mlsanity.com/paper/2510.01976/) (2025)
- [Rules Before Oracles: Auditable, User-Configurable Argument Selection for Deliberative Polling](https://mlsanity.com/paper/2608.23979/) (2026)

## Abstract (arXiv)

Understanding citizens' values in participatory systems is crucial for citizen-centric policy-making. We envision a hybrid participatory system where participants make choices and provide motivations for those choices, and AI agents estimate their value preferences by interacting with them. We focus on situations where a conflict is detected between participants' choices and motivations, and propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants. We operationalize the philosophical stance that "valuing is deliberatively consequential." That is, if a participant's choice is based on a deliberation of value preferences, the value preferences can be observed in the motivation the participant provides for the choice. Thus, we propose and compare value preferences estimation methods that prioritize the values estimated from motivations over the values estimated from choices alone. Then, we introduce a disambiguation strategy that combines Natural Language Processing and Active Learning to address the detected inconsistencies between choices and motivations. We evaluate the proposed methods on a dataset of a large-scale survey on energy transition. The results show that explicitly addressing inconsistencies between choices and motivations improves the estimation of an individual's value preferences. The disambiguation strategy does not show substantial improvements when compared to similar baselines--however, we discuss how the novelty of the approach can open new research avenues and propose improvements to address the current limitations.