# Jury Learning: Integrating Dissenting Voices into Machine Learning Models

arXiv: https://arxiv.org/abs/2202.02950 · HTML: https://mlsanity.com/paper/2202.02950/

## Key facts

- Cited by 118 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 97 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [On Releasing Annotator-Level Labels and Information in Datasets](https://mlsanity.com/paper/2110.05699/) (2021)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development](https://mlsanity.com/paper/2108.04308/) (2021)
- [Designing Toxic Content Classification for a Diversity of Perspectives](https://mlsanity.com/paper/2106.04511/) (2021)
- [Challenges for Toxic Comment Classification: An In-Depth Error Analysis](https://mlsanity.com/paper/1809.07572/) (2018)
- [Changing the World by Changing the Data](https://mlsanity.com/paper/2105.13947/) (2021)
- [Roles for Computing in Social Change](https://mlsanity.com/paper/1912.04883/) (2019)
- [Screen2Vec: Semantic Embedding of GUI Screens and GUI Components](https://mlsanity.com/paper/2101.11103/) (2021)
- [Whose Opinions Matter? Perspective-aware Models to Identify Opinions of Hate Speech Victims in Abusive Language Detection](https://mlsanity.com/paper/2106.15896/) (2021)
- [Society-in-the-Loop: Programming the Algorithmic Social Contract](https://mlsanity.com/paper/1707.07232/) (2017)
- [A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities](https://mlsanity.com/paper/1812.00315/) (2018)
- [Convolutional Neural Networks for Toxic Comment Classification](https://mlsanity.com/paper/1802.09957/) (2018)
- [Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems](https://mlsanity.com/paper/2001.04879/) (2020)
- [PolicyKit: Building Governance in Online Communities](https://mlsanity.com/paper/2008.04236/) (2020)
- [Personalized Embedding-based e-Commerce Recommendations at eBay](https://mlsanity.com/paper/2102.06156/) (2021)
- [Crowdsourcing Ground Truth for Medical Relation Extraction](https://mlsanity.com/paper/1701.02185/) (2017)
- [Capturing Ambiguity in Crowdsourcing Frame Disambiguation](https://mlsanity.com/paper/1805.00270/) (2018)
- [ESR: Ethics and Society Review of Artificial Intelligence Research](https://mlsanity.com/paper/2106.11521/) (2021)
- [Crowdsourcing for Participatory Democracies: Efficient Elicitation of Social Choice Functions](https://mlsanity.com/paper/1406.7542/) (2014)

## Built on by (most cited first, 30 of 118)

- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [We're Afraid Language Models Aren't Modeling Ambiguity](https://mlsanity.com/paper/2304.14399/) (2023)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties](https://mlsanity.com/paper/2309.00779/) (2023)
- [When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks](https://mlsanity.com/paper/2305.06626/) (2023)
- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [Human Feedback is not Gold Standard](https://mlsanity.com/paper/2309.16349/) (2023)
- [PAD: Personalized Alignment of LLMs at Decoding-Time](https://mlsanity.com/paper/2410.04070/) (2024)
- [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://mlsanity.com/paper/2305.00955/) (2023)
- [The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values](https://mlsanity.com/paper/2310.07629/) (2023)
- [Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information](https://mlsanity.com/paper/2301.05036/) (2023)
- [Beyond Preferences in AI Alignment](https://mlsanity.com/paper/2408.16984/) (2024)
- [The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics](https://mlsanity.com/paper/2306.11559/) (2023)
- [Language Model Alignment in Multilingual Trolley Problems](https://mlsanity.com/paper/2407.02273/) (2024)
- [Eliciting and Learning with Soft Labels from Every Annotator](https://mlsanity.com/paper/2207.00810/) (2022)
- [You Are What You Annotate: Towards Better Models through Annotator Representations](https://mlsanity.com/paper/2305.14663/) (2023)
- [HumanLM: Simulating Users with State Alignment Beats Response Imitation](https://mlsanity.com/paper/2603.03303/) (2026)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory](https://mlsanity.com/paper/2205.05057/) (2022)
- [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022)
- [Conformal prediction under ambiguous ground truth](https://mlsanity.com/paper/2307.09302/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)

## Abstract (arXiv)

Whose labels should a machine learning (ML) algorithm learn to emulate? For ML tasks ranging from online comment toxicity to misinformation detection to medical diagnosis, different groups in society may have irreconcilable disagreements about ground truth labels. Supervised ML today resolves these label disagreements implicitly using majority vote, which overrides minority groups' labels. We introduce jury learning, a supervised ML approach that resolves these disagreements explicitly through the metaphor of a jury: defining which people or groups, in what proportion, determine the classifier's prediction. For example, a jury learning model for online toxicity might centrally feature women and Black jurors, who are commonly targets of online harassment. To enable jury learning, we contribute a deep learning architecture that models every annotator in a dataset, samples from annotators' models to populate the jury, then runs inference to classify. Our architecture enables juries that dynamically adapt their composition, explore counterfactuals, and visualize dissent.