mlsanityData to 14 Oct 2026

Do LLMs have Consistent Values?

arXiv 2407.12878 ↗First posted 16 Jul 2024cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLM) technology is constantly improving towards human-like dialogue. Values are a basic driving force underlying human behavior, but little research has been done to study the values exhibited in text generated by LLMs. Here we study this question by turning to the rich literature on value structure in psychology. We ask whether LLMs exhibit the same value structure that has been demonstrated in humans, including the ranking of values, and correlation between values. We show that the results of this analysis depend on how the LLM is prompted, and that under a particular prompting strategy (referred to as "Value Anchoring") the agreement with human data is quite compelling. Our results serve both to improve our understanding of values in LLMs, as well as introduce novel methods for assessing consistency in LLM responses.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 27 of its 66 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

27 of 66 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  3. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  4. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  5. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  6. 2022Using cognitive psychology to understand GPT-3cited by 398
  7. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  8. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
  9. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  10. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  11. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  12. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 179
15 more
  1. 2023Personality Traits in Large Language Modelscited by 169
  2. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  3. 2022Thinking Fast and Slow in Large Language Modelscited by 137
  4. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 135
  5. 2022Who is GPT-3? An Exploration of Personality, Values and Demographicscited by 112
  6. 2022Putting GPT-3's Creativity to the (Alternative Uses) Testcited by 62
  7. 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 60
  8. 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 53
  9. 2023Large Language Models as Superpositions of Cultural Perspectivescited by 49
  10. 2023What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theorycited by 19
  11. 2024ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Modelscited by 19
  12. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  13. 2021Towards Socially Intelligent Agents with Mental State Transition and Human Utilitycited by 14
  14. 2024Stick to your Role! Stability of Personal Values Expressed in Large Language Modelscited by 14
  15. 2023From Values to Opinions: Predicting Human Behaviors and Stances Using Value-Injected Large Language Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
  2. 2025Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmascited by 18
  3. 2026FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insightscited by 16
  4. 2026Knowing But Not Doing: Convergent Morality and Divergent Action in LLMscited by 7
  5. 2025Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Modelscited by 4
  6. 2026Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuumcited by 3
  7. 2026Do Schwartz Higher-Order Values Help Sentence-Level Human Value Detection? A Study of Hierarchical Gating and Calibrationcited by 3
  8. 2026Measuring Opinion Bias and Sycophancy via LLM-based Persuasioncited by 3
  9. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  10. 2026How AI Systems Think About Education: Analyzing Latent Preference Patterns in Large Language Modelscited by 1
  11. 2026Superficial Beliefs in LLM Decision-Makingcited by 0
  12. 2026Occupational Prompting Reveals Cultural Bias in Large Language Modelscited by 0
2 more
  1. 2026PrimeScientist: Strategic Allocation of Research Effort in Autonomous Researchcited by 0
  2. 2026Benchmarking Psychological Dynamics in Generative Agentscited by 0