# Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset

arXiv: https://arxiv.org/abs/2507.09650 · HTML: https://mlsanity.com/paper/2507.09650/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 34 of its 100 references that are in the map.
- First posted on arXiv 13 Jul 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 30 of 34)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging](https://mlsanity.com/paper/2310.11564/) (2023)
- [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://mlsanity.com/paper/2406.09948/) (2024)
- [Personalized Language Modeling from Personalized Human Feedback](https://mlsanity.com/paper/2402.05133/) (2024)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [What Makes a Reward Model a Good Teacher? An Optimization Perspective](https://mlsanity.com/paper/2503.15477/) (2025)
- [Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models](https://mlsanity.com/paper/2310.13102/) (2023)
- [Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking](https://mlsanity.com/paper/2402.09369/) (2024)
- [Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs](https://mlsanity.com/paper/2503.08688/) (2025)
- [Diverse Preference Optimization](https://mlsanity.com/paper/2501.18101/) (2025)
- [Language Model Alignment in Multilingual Trolley Problems](https://mlsanity.com/paper/2407.02273/) (2024)
- [Everyone Deserves A Reward: Learning Customized Human Preferences](https://mlsanity.com/paper/2309.03126/) (2023)
- [Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.16534/) (2025)
- [LoRe: Personalizing LLMs via Low-Rank Reward Modeling](https://mlsanity.com/paper/2504.14439/) (2025)
- [Is a Good Foundation Necessary for Efficient Reinforcement Learning? The Computational Role of the Base Model in Exploration](https://mlsanity.com/paper/2503.07453/) (2025)
- [Assessing LLMs for Moral Value Pluralism](https://mlsanity.com/paper/2312.10075/) (2023)
- [The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models](https://mlsanity.com/paper/2502.11266/) (2025)
- [Aligning Crowd Feedback via Distributional Preference Reward Modeling](https://mlsanity.com/paper/2402.09764/) (2024)
- [Generative Monoculture in Large Language Models](https://mlsanity.com/paper/2407.02209/) (2024)
- [Cultural Incongruencies in Artificial Intelligence](https://mlsanity.com/paper/2211.13069/) (2022)
- [Democratic Policy Development using Collective Dialogues and AI](https://mlsanity.com/paper/2311.02242/) (2023)
- [Jackpot! Alignment as a Maximal Lottery](https://mlsanity.com/paper/2501.19266/) (2025)
- [CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models](https://mlsanity.com/paper/2405.13974/) (2024)
- ['Generative CI' through Collective Response Systems](https://mlsanity.com/paper/2302.00672/) (2023)
- [The Value of Disagreement in AI Design, Evaluation, and Alignment](https://mlsanity.com/paper/2505.07772/) (2025)

## Built on by (most cited first, 30 of 35)

- [What's In My Human Feedback? Learning Interpretable Descriptions of Preference Data](https://mlsanity.com/paper/2510.26202/) (2025)
- [Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models](https://mlsanity.com/paper/2507.13383/) (2025)
- [Value Drifts: Tracing Value Alignment During LLM Post-Training](https://mlsanity.com/paper/2510.26707/) (2025)
- [Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior](https://mlsanity.com/paper/2511.14476/) (2025)
- [Strategic Algorithmic Monoculture: Experimental Evidence from Coordination Games](https://mlsanity.com/paper/2604.09502/) (2026)
- [PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm](https://mlsanity.com/paper/2601.08951/) (2026)
- [Positive Alignment: Artificial Intelligence for Human Flourishing](https://mlsanity.com/paper/2605.10310/) (2026)
- [Identity-Aware Large Language Models require Cultural Reasoning](https://mlsanity.com/paper/2510.18510/) (2025)
- [EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferences](https://mlsanity.com/paper/2510.06370/) (2025)
- [PERSPECTRA: A Scalable and Configurable Pluralist Benchmark of Perspectives from Arguments](https://mlsanity.com/paper/2602.08716/) (2026)
- [Parametric Social Identity Injection and Diversification in Public Opinion Simulation](https://mlsanity.com/paper/2603.16142/) (2026)
- [Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers](https://mlsanity.com/paper/2606.02776/) (2026)
- [Humans and LLMs Diverge on Probabilistic Inferences](https://mlsanity.com/paper/2602.23546/) (2026)
- [Automated Concept Discovery for LLM-as-a-Judge Preference Analysis](https://mlsanity.com/paper/2603.03319/) (2026)
- [SCRuB: Social Concept Reasoning under Rubric-Based Evaluation](https://mlsanity.com/paper/2605.06444/) (2026)
- [PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users](https://mlsanity.com/paper/2605.13307/) (2026)
- [Learning to Route Languages for Multilingual Policy Optimization](https://mlsanity.com/paper/2605.25360/) (2026)
- [Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth](https://mlsanity.com/paper/2607.00139/) (2026)
- [Mitigating Identity Essentialism in LLM Agents with Longitudinal Life Trajectories](https://mlsanity.com/paper/2608.19621/) (2026)
- [Distortion of AI Alignment Revisited: RLHF is a Decent Utilitarian Aligner](https://mlsanity.com/paper/2609.12651/) (2026)
- [CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters](https://mlsanity.com/paper/2601.04885/) (2026)
- [How Value Induction Reshapes LLM Behaviour](https://mlsanity.com/paper/2605.07925/) (2026)
- [Quantifying the Salience of Geo-Cultural Values for Pluralistic Safety Alignment](https://mlsanity.com/paper/2606.00369/) (2026)
- [Pigeonholing: how bad prompts hurt models, causing collapse and mistakes](https://mlsanity.com/paper/2606.24267/) (2026)
- [PEBS: Per-rater Empirical-Bayes Shrinkage for RLHF Reward-Model Calibration](https://mlsanity.com/paper/2606.27578/) (2026)
- [A Roadmap to Impactful Pluralistic Alignment Research](https://mlsanity.com/paper/2607.22305/) (2026)
- [What do Reward Models Memorize?](https://mlsanity.com/paper/2607.24484/) (2026)
- [PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs](https://mlsanity.com/paper/2608.01458/) (2026)
- [Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generation](https://mlsanity.com/paper/2608.05576/) (2026)
- [Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment](https://mlsanity.com/paper/2608.11528/) (2026)

## Abstract (arXiv)

How can large language models (LLMs) serve users with varying preferences that may conflict across cultural, political, or other dimensions? To advance this challenge, this paper establishes four key results. First, we demonstrate, through a large-scale multilingual human study with representative samples from five countries (N=15,000), that humans exhibit substantially more variation in preferences than the responses of 21 state-of-the-art LLMs. Second, we show that existing methods for preference dataset collection are insufficient for learning the diversity of human preferences even along two of the most salient dimensions of variability in global values, due to the underlying homogeneity of candidate responses. Third, we argue that this motivates the need for negatively-correlated sampling when generating candidate sets, and we show that simple prompt-based techniques for doing so greatly enhance the performance of alignment methods in learning heterogeneous preferences. Fourth, based on this novel candidate sampling approach, we collect and open-source Community Alignment} the largest and most representative multilingual and multi-turn preference dataset to date, featuring 233,319 comparisons from annotators spanning five countries. Overall, we hope that the Community Alignment dataset will be a valuable resource for improving the effectiveness of LLMs for a diverse global population.