mlsanityData to 14 Oct 2026

Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions

arXiv 2504.15236 ↗First posted 21 Apr 2025cs.CL cs.AI cs.CYcited by 52

Not on the map. Why?

Abstract (arXiv)

AI assistants can impart value judgments that shape people's decisions and worldviews, yet little is known empirically about what values these systems rely on in practice. To address this, we develop a bottom-up, privacy-preserving method to extract the values (normative considerations stated or demonstrated in model responses) that Claude 3 and 3.5 models exhibit in hundreds of thousands of real-world interactions. We empirically discover and taxonomize 3,307 AI values and study how they vary by context. We find that Claude expresses many practical and epistemic values, and typically supports prosocial human values while resisting values like "moral nihilism". While some values appear consistently across contexts (e.g. "transparency"), many are more specialized and context-dependent, reflecting the diversity of human interlocutors and their varied contexts. For example, "harm prevention" emerges when Claude resists users, "historical accuracy" when responding to queries about controversial events, "healthy boundaries" when asked for relationship advice, and "human agency" in technology ethics discussions. By providing the first large-scale empirical mapping of AI values in deployment, our work creates a foundation for more grounded evaluation and design of values in AI systems.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 25 of its 52 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

25 of 52 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  3. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  5. 2023Whose Opinions Do Language Models Reflect?cited by 773
  6. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  7. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  8. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  9. 2020Artificial Intelligence, Values and Alignmentcited by 448
  10. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  11. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  12. 2023Personality Traits in Large Language Modelscited by 169
13 more
  1. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  2. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  3. 2022Evaluating and Inducing Personality in Pre-trained Language Modelscited by 123
  4. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
  5. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  6. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  7. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  8. 2024Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooMcited by 78
  9. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  10. 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
  11. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
  12. 2023Large Language Models as Superpositions of Cultural Perspectivescited by 49
  13. 2024Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 22; 2026: 30.

52 papers.

Sort:

By area:

  1. 2025What's In My Human Feedback? Learning Interpretable Descriptions of Preference Datacited by 26
  2. 2025Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmascited by 18
  3. 2025The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmascited by 13
  4. 2025Generative Value Conflicts Reveal LLM Prioritiescited by 13
  5. 2025Stress-Testing Model Specs Reveals Character Differences among Language Modelscited by 13
  6. 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 10
  7. 2025MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanationcited by 8
  8. 2025The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Modelscited by 7
  9. 2026Knowing But Not Doing: Convergent Morality and Divergent Action in LLMscited by 7
  10. 2025Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Valuecited by 6
  11. 2025Real-World Gaps in AI Governance Researchcited by 5
  12. 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
38 more
  1. 2025Value Alignment of Social Media Ranking Algorithmscited by 4
  2. 2026Evaluation Cards: An Interpretive Layer for AI Evaluation Reportingcited by 4
  3. 2026Persona Cartography: Charting Language Model Personality Traits in Weight Spacecited by 4
  4. 2025Show or Tell? Modeling the evolution of request-making in Human-LLM conversationscited by 3
  5. 2025Decoding Alignment: A Critical Survey of LLM Development Initiatives through Value-setting and Data-centric Lenscited by 3
  6. 2025Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarizationcited by 3
  7. 2025Interaction Protocol Shapes Moral Judgment in Multi-Agent Debatecited by 3
  8. 2025Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theorycited by 3
  9. 2025Value Lens: Using Large Language Models to Understand Human Valuescited by 3
  10. 2026VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Modelscited by 3
  11. 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
  12. 2025Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Valuescited by 2
  13. 2026Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impactcited by 2
  14. 2026Reflections and New Directions for Human-Centered Large Language Modelscited by 2
  15. 2025Black Box Absorption: LLMs Undermining Innovative Ideascited by 1
  16. 2026Growth First, Care Second? Tracing the Landscape of LLM Value Preferences in Everyday Dilemmascited by 1
  17. 2026Abstractive Red-Teaming of Language Model Charactercited by 1
  18. 2026Automated Concept Discovery for LLM-as-a-Judge Preference Analysiscited by 1
  19. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  20. 2026Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebookcited by 1
  21. 2026KVpop -- Key-Value Cache Compression with Predictive Online Pruningcited by 1
  22. 2026PLURAL: A Global Dataset for Value Alignmentcited by 1
  23. 2026Conduct Under Pressure: What Sixty Language Models Do When a User Pushescited by 1
  24. 2025NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Contextcited by 0
  25. 2025'Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMscited by 0
  26. 2025VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Modelscited by 0
  27. 2026Discovering Implicit Large Language Model Alignment Objectivescited by 0
  28. 2026Mechanistic Origin of Moral Indifference in Language Modelscited by 0
  29. 2026Does Claude's Constitution Have a Culture?cited by 0
  30. 2026"What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Usecited by 0
  31. 2026Theoretical Limits of Language Model Alignmentcited by 0
  32. 2026How Value Induction Reshapes LLM Behaviourcited by 0
  33. 2026Identifying and Understanding Human Values in Text: A Tailorable LLM-based Architecturecited by 0
  34. 2026RobotValues: Evaluating Household Robots When Human Values Conflictcited by 0
  35. 2026DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Modelscited by 0
  36. 2026From Behavior to Mechanism: Tracing Divergent Response Modes in Frontier Language Modelscited by 0
  37. 2026Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperationcited by 0
  38. 2026CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asiacited by 0

50 most cited of 52 All 52 (JSON)