mlsanityData to 14 Oct 2026

Can Language Models Reason about Individualistic Human Values and Preferences?

arXiv 2410.03868 ↗First posted 4 Oct 2024cs.CLcited by 34

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Recent calls for pluralistic alignment emphasize that AI systems should address the diverse needs of all people. Yet, efforts in this space often require sorting people into fixed buckets of pre-specified diversity-defining dimensions (e.g., demographics), risking smoothing out individualistic variations or even stereotyping. To achieve an authentic representation of diversity that respects individuality, we propose individualistic alignment. While individualistic alignment can take various forms, we introduce IndieValueCatalog, a dataset transformed from the influential World Values Survey (WVS), to study language models (LMs) on the specific challenge of individualistic value reasoning. Given a sample of an individual's value-expressing statements, models are tasked with predicting this person's value judgments in novel cases. With IndieValueCatalog, we reveal critical limitations in frontier LMs, which achieve only 55 % to 65% accuracy in predicting individualistic values. Moreover, our results highlight that a precise description of individualistic values cannot be approximated only with demographic information. We also identify a partiality of LMs in reasoning about global individualistic values, as measured by our proposed Value Inequity Index (σInequity). Finally, we train a series of IndieValueReasoners to reveal new patterns and dynamics into global human values.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 33 of its 60 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

33 of 60 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  6. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  7. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  8. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  9. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 280
  10. 2024A Roadmap to Pluralistic Alignmentcited by 184
  11. 2023Personality Traits in Large Language Modelscited by 169
  12. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 155
21 more
  1. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  2. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 113
  3. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  4. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 92
  5. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  6. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignmentcited by 73
  7. 2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignmentcited by 63
  8. 2024Active Preference Learning for Large Language Modelscited by 58
  9. 2024PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferencescited by 54
  10. 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 51
  11. 2024Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinementcited by 49
  12. 2024Value Augmented Sampling for Language Model Alignment and Personalizationcited by 49
  13. 2024Reinforcement Learning from Human Feedback with Active Queriescited by 43
  14. 2024Personality Alignment of Large Language Modelscited by 36
  15. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  16. 2024LLMs + Persona-Plug = Personalized LLMscited by 32
  17. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 30
  18. 2023Active Preference Inference using Language Models and Probabilistic Reasoningcited by 29
  19. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  20. 2023Sample Efficient Preference Alignment in LLMs via Active Explorationcited by 18
  21. 2024On the Pros and Cons of Active Learning for Moral Preference Elicitationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 15; 2026: 18.

34 papers.

Sort:

By area:

  1. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  2. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 66
  3. 2025Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?cited by 51
  4. 2025Value Profiles for Encoding Human Variationcited by 29
  5. 2025Direct Alignment with Heterogeneous Preferencescited by 19
  6. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  7. 2025CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMscited by 12
  8. 2025CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectivescited by 10
  9. 2025From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMscited by 9
  10. 2026Learning User Simulators with Turing Rewardscited by 7
  11. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6
  12. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 5
22 more
  1. 2026PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harmcited by 5
  2. 2025Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Modelscited by 4
  3. 2026ValueFlow: Measuring the Propagation of Value Perturbations in Multi-Agent LLM Systemscited by 4
  4. 2026CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientistscited by 4
  5. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferencescited by 3
  6. 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Testcited by 2
  7. 2025ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Makingcited by 2
  8. 2026What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Modelscited by 2
  9. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  10. 2025Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?cited by 1
  11. 2026Value Alignment Tax: Measuring Value Trade-offs in LLM Alignmentcited by 1
  12. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  13. 2026Spectral Souping: A Unified Framework for Online Preference Alignmentcited by 1
  14. 2025Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertisecited by 0
  15. 2026CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapterscited by 0
  16. 2026Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLPcited by 0
  17. 2026The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Predictioncited by 0
  18. 2026CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queriescited by 0
  19. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0
  20. 2026ExpertIVS: Sociological Expert Driven Individual Value Simulation in Large Language Modelscited by 0
  21. 2026Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptationcited by 0
  22. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models
  2. 2023Heterogeneous Value Alignment Evaluation for Large Language Models
  3. 2025Diverse Human Value Alignment for Large Language Models via Ethical Reasoning
  4. 2026Toward a Theory of Value in AI Alignment
  5. 2025Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires
  6. 2025Cultural Value Alignment in Large Language Models: A Prompt-based Analysis of Schwartz Values in Gemini, ChatGPT, and DeepSeek
  7. 2026Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences
  8. 2026Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models