mlsanityData to 14 Oct 2026

ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models

arXiv 2406.04214 ↗First posted 6 Jun 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are transforming diverse fields and gaining increasing influence as human proxies. This development underscores the urgent need for evaluating value orientations and understanding of LLMs to ensure their responsible integration into public-facing applications. This work introduces ValueBench, the first comprehensive psychometric benchmark for evaluating value orientations and value understanding in LLMs. ValueBench collects data from 44 established psychometric inventories, encompassing 453 multifaceted value dimensions. We propose an evaluation pipeline grounded in realistic human-AI interactions to probe value orientations, along with novel tasks for evaluating value understanding in an open-ended value space. With extensive experiments conducted on six representative LLMs, we unveil their shared and distinctive value orientations and exhibit their ability to approximate expert conclusions in value-related extraction and generation tasks. ValueBench is openly accessible at https://github.com/Value4AI/ValueBench.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 16 of its 104 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

16 of 104 references.

References · 16

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  5. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  6. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  7. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  8. 2022Who is GPT-3? An Exploration of Personality, Values and Demographicscited by 112
  9. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 79
  10. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 74
  11. 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
  12. 2023Can ChatGPT Assess Human Personalities? A General Evaluation Frameworkcited by 70
4 more
  1. 2022Estimating the Personality of White-Box Language Modelscited by 48
  2. 2022Identifying and Manipulating the Personality Traits of Language Modelscited by 40
  3. 2023Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMscited by 32
  4. 2023Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimationcited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 10; 2026: 6.

19 papers.

Sort:

By area:

  1. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 119
  2. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
  3. 2024Large Language Models Reflect the Ideology of their Creatorscited by 57
  4. 2024Do LLMs have Consistent Values?cited by 14
  5. 2024Measuring Human and AI Values Based on Generative Psychometrics with Large Language Modelscited by 13
  6. 2026Evaluating Alignment of Behavioral Dispositions in LLMscited by 7
  7. 2025Deep Value Benchmark: Measuring Whether Models Generalize Deep Values or Shallow Preferencescited by 6
  8. 2026ScienceDB AI: An LLM-Driven Agentic Recommender System for Large-Scale Scientific Data Sharing Servicescited by 6
  9. 2025EAVIT: Efficient and Accurate Human Value Identification from Text data via LLMscited by 5
  10. 2025Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Modelscited by 4
  11. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  12. 2025Interaction Protocol Shapes Moral Judgment in Multi-Agent Debatecited by 3
7 more
  1. 2025Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengthscited by 2
  2. 2025NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Contextcited by 0
  3. 2025The Realignment Problem: When Right becomes Wrong in LLMscited by 0
  4. 2026Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmarkcited by 0
  5. 2026Bridging Values and Behavior: A Hierarchical Framework for Proactive Embodied Agentscited by 0
  6. 2026Which Values Do LLMs Confuse? A Schwartz-Based Recognition Studycited by 0
  7. 2026Values as Style: Disentangling Values from Semantics with One-Way Mixing for Low-Damage LLM Steeringcited by 0