Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Large language models (LLMs) are increasingly used in culturally sensitive settings, where alignment requires representing diverse preferences within populations. Yet existing methods model populations at coarse demographic or community levels and overlook within-group variation. We introduce Demographic Pluralism, an inference-time framework that estimates population-level opinion distributions without opinion-distribution training data or task-specific fine-tuning by generating multiple perspectives within demographically grounded groups. Across four backbones on GlobalOpinionQA and VITAL, it reduces Jensen-Shannon distance by 8.4%-26.4% over Modular Pluralism. Among weighted, equal-weighted, and inverse-weighted aggregation, equal weighting performs best overall; group-level error also increases with group weight, helping explain weighted aggregation's weaker performance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 36 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
21 of 36 references.
Closest ancestors · 7
Same direction, newest first
- 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset13 Jul
- 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuning20 Mar
- 2025PluralLLM: Pluralistic Alignment in LLMs via Federated Learning13 Mar
- 2024PERSONA: A Reproducible Testbed for Pluralistic Alignment24 Jul
- 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration22 Jun
- 2024Collective Constitutional AI: Aligning a Language Model with Public Input12 Jun
- 2024Investigating Cultural Alignment of Large Language Models20 Feb
Other references · 14
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
- 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 123
- 2024PersonaGym: Evaluating Persona Agents and LLMscited by 97
- 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 53
- 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 51
- 2023Faithful Persona-based Conversational Dataset Generation with Large Language Modelscited by 47
- 2025Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?cited by 22
- 2025VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcarecited by 17
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Unpacking Human Preference for LLMs: Demographically Aware Evaluation with the HUMAINE Framework
- 2024Benchmarking Distributional Alignment of Large Language Models
- 2024No Preference Left Behind: Group Distributional Preference Optimization
- 2024A Roadmap to Pluralistic Alignment
- 2026Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
- 2026Hidden Consensus:Preference-Validity Compression in Human Feedback
- 2025Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior
- 2026Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment