mlsanityData to 7 Oct 2026

Value Profiles for Encoding Human Variation

arXiv 2503.15484 ↗First posted 19 Mar 2025cs.CL cs.AI cs.HCcited by 29

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Modelling human variation in rating tasks is crucial for personalization, pluralistic model alignment, and computational social science. We propose representing individuals using natural language value profiles -- descriptions of underlying values compressed from in-context demonstrations -- along with a steerable decoder model that estimates individual ratings from a rater representation. To measure the predictive information in a rater representation, we introduce an information-theoretic methodology and find that demonstrations contain the most information, followed by value profiles, then demographics. However, value profiles effectively compress the useful information from demonstrations (>70% information preservation) and offer advantages in terms of scrutability, interpretability, and steerability. Furthermore, clustering value profiles to identify similarly behaving individuals better explains rater variation than the most predictive demographic groupings. Going beyond test set performance, we show that the decoder predictions change in line with semantic profile differences, are well-calibrated, and can help explain instance-level disagreement by simulating an annotator population. These results demonstrate that value profiles offer novel, predictive ways to describe individual variation beyond demographics or group information.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 36 of its 57 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

36 of 57 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Diverging Preferences: When do Annotators Disagree and do Models Know?18 Oct
  2. 2024Can Language Models Reason about Individualistic Human Values and Preferences?4 Oct
  3. 2024Inverse Constitutional AI: Compressing Preferences into Principles2 Jun
  4. 2024What are human values, and how do we align AI to them?27 Mar
  5. 2023Aligning Language Models to User Opinions24 May

Other references · 31

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  6. 2023Whose Opinions Do Language Models Reflect?cited by 759
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  8. 2020Artificial Intelligence, Values and Alignmentcited by 443
  9. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  10. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
  11. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  12. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
19 more
  1. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  2. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
  3. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 137
  4. 2023Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHFcited by 125
  5. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  6. 2021On Measures of Biases and Harms in NLPcited by 81
  7. 2021Designing Toxic Content Classification for a Diversity of Perspectivescited by 63
  8. 2023On the steerability of large language models toward data-driven personascited by 53
  9. 2023The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographicscited by 45
  10. 2021Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communitiescited by 40
  11. 2021Early-stopped neural networks are consistentcited by 34
  12. 2025Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptionscited by 31
  13. 2024Transparent and Scrutable Recommendations Using Natural Language User Profilescited by 24
  14. 2022On Natural Language User Profiles for Transparent and Scrutable Recommendationcited by 23
  15. 2023Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreementcited by 22
  16. 2024Supernotes: Driving Consensus in Crowd-Sourced Fact-Checkingcited by 22
  17. 2023GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectivescited by 16
  18. 2024Voices in a Crowd: Searching for Clusters of Unique Perspectivescited by 9
  19. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 15.

29 papers.

Sort:

By area:

  1. 2025Pairwise Calibrated Rewards for Pluralistic Alignmentcited by 12
  2. 2025Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Modelscited by 11
  3. 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 9
  4. 2025Resource Rational Contractualism Should Guide AI Alignmentcited by 8
  5. 2025Benchmarking Overton Pluralism in LLMscited by 7
  6. 2025Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Valuecited by 6
  7. 2026PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harmcited by 5
  8. 2025Decoding Alignment: A Critical Survey of LLM Development Initiatives through Value-setting and Data-centric Lenscited by 3
  9. 2025From Noise to Signal to Selbstzweck: Reframing Human Label Variation in the Era of Post-training in NLPcited by 3
  10. 2026VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Modelscited by 3
  11. 2026AI Alignment Breaks at the Edgecited by 3
  12. 2025Improving the Distributional Alignment of LLMs using Supervisioncited by 2
17 more
  1. 2025Minding the Politeness Gap in Cross-cultural Communicationcited by 1
  2. 2025Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learningcited by 1
  3. 2026Labels have Human Values: Value Calibration of Subjective Taskscited by 1
  4. 2025SOLAR: Towards Characterizing Subjectivity of Individuals through Modeling Value Conflicts and Trade-offscited by 0
  5. 2025DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learningcited by 0
  6. 2025Hypernetworks for Perspectivist Adaptationcited by 0
  7. 2026Robust Persona-Aware Toxicity Detection with Prompt Optimization and Learned Ensemblingcited by 0
  8. 2026EpiPersona: Persona Projection and Episode Coupling for Pluralistic Preference Modelingcited by 0
  9. 2026Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLPcited by 0
  10. 2026Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEMcited by 0
  11. 2026When Does Demographic Information Help? Data and Modeling Regimes for Perspective-Aware Hate Speech Detectioncited by 0
  12. 2026Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimizationcited by 0
  13. 2026AI Alignment From Social Choice Perspectivescited by 0
  14. 2026Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactionscited by 0
  15. 2026LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurtscited by 0
  16. 2026Toward Collective-Centric Evaluation of Preference Inference for Participatory Democracycited by 0
  17. 2026Multi-Dimensional Comparative Scale Construction for Efficient Personalized Subjective Judgment in High-Traffic Applicationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferences
  2. 2026Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
  3. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignment
  4. 2026VRM: Teaching Reward Models to Understand Authentic Human Preferences
  5. 2025Survey-to-Behavior: Downstream Alignment of Human Values in LLMs via Survey Questions
  6. 2025DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
  7. 2023DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4
  8. 2026Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization