# Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths

arXiv: https://arxiv.org/abs/2506.02481 · HTML: https://mlsanity.com/paper/2506.02481/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 27 of 27)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards](https://mlsanity.com/paper/2402.18571/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Verbosity Bias in Preference Labeling by Large Language Models](https://mlsanity.com/paper/2310.10076/) (2023)
- [Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity](https://mlsanity.com/paper/2209.12106/) (2022)
- [DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life](https://mlsanity.com/paper/2410.02683/) (2024)
- [Are Large Language Models Consistent over Value-laden Questions?](https://mlsanity.com/paper/2407.02996/) (2024)
- [Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions](https://mlsanity.com/paper/2410.02584/) (2024)
- [Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores](https://mlsanity.com/paper/2403.00553/) (2024)
- [Large Language Models as Superpositions of Cultural Perspectives](https://mlsanity.com/paper/2307.07870/) (2023)
- [You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments](https://mlsanity.com/paper/2311.09718/) (2023)
- [From Distributional to Overton Pluralism: Investigating Large Language Model Alignment](https://mlsanity.com/paper/2406.17692/) (2024)
- [Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs](https://mlsanity.com/paper/2305.14693/) (2023)
- [Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility](https://mlsanity.com/paper/2305.10235/) (2023)
- [Evaluating Large Language Models with Psychometrics](https://mlsanity.com/paper/2406.17675/) (2024)
- [Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models](https://mlsanity.com/paper/2401.07115/) (2024)
- [Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models](https://mlsanity.com/paper/2402.13887/) (2024)
- [ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models](https://mlsanity.com/paper/2406.04214/) (2024)
- [Evaluating LLM Metrics Through Real-World Capabilities](https://mlsanity.com/paper/2505.08253/) (2025)
- [Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches](https://mlsanity.com/paper/2404.12744/) (2024)
- [The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning](https://mlsanity.com/paper/2405.12744/) (2024)
- [Determining Relative Argument Specificity and Stance for Complex Argumentative Structures](https://mlsanity.com/paper/1906.11313/) (2019)
- [Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Values](https://mlsanity.com/paper/2501.07071/) (2025)

## Built on by (most cited first, 2 of 2)

- [Asymmetric Goal Drift in Coding Agents Under Value Conflict](https://mlsanity.com/paper/2603.03456/) (2026)
- [Belief-Sim: Towards Belief-Driven Simulation of Demographic Misinformation Susceptibility](https://mlsanity.com/paper/2603.03585/) (2026)

## Abstract (arXiv)

Evaluations of LLMs' ethical risks and value inclinations often rely on short-form surveys and psychometric tests, yet real-world use involves long-form, open-ended responses -- leaving value-related risks and preferences in practical settings largely underexplored. In this work, we ask: Do value preferences inferred from short-form tests align with those expressed in long-form outputs? To address this question, we compare value preferences elicited from short-form reactions and long-form responses, varying the number of arguments in the latter to capture users' differing verbosity preferences. Analyzing five LLMs (llama3-8b, gemma2-9b, mistral-7b, qwen2-7b, and olmo-7b), we find (1) a weak correlation between value preferences inferred from short-form and long-form responses across varying argument counts, and (2) similarly weak correlation between preferences derived from any two distinct long-form generation settings. (3) Alignment yields only modest gains in the consistency of value expression. Further, we examine how long-form generation attributes relate to value preferences, finding that argument specificity negatively correlates with preference strength, while representation across scenarios shows a positive correlation. Our findings underscore the need for more robust methods to ensure consistent value expression across diverse applications.