mlsanityData to 7 Oct 2026

AI Revealed Preferences

arXiv 2608.26178 ↗First posted 17 Aug 2026cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

There is growing interest in whether language models have stable preferences, for technical, safety, and philosophical reasons. We test 20 language models and find a range of preferences---stable dispositions to choose certain kinds of tasks. We run three forced-choice experiments on revealed rather than stated preferences, requiring models not only to rank tasks, but to actually perform them. Headline findings include evidence that models are tedium-averse, "leisure"-seeking, and covertly sycophantic. Tedium aversion means that, when tasks are tedious (alphabetization), models choose shorter tasks than when tasks are creative (generating metaphors). "Leisure"-seeking describes models' preference for tasks whose ideal answers match what they produce when left to write freely. Covert sycophancy means that models avoid answering questions where an honest response would be unwelcome, even if helpful. Beyond these results, we find convergent cross-model preferences over occupations drawn from the GDPval benchmark (technical jobs over real estate), over question types (concept explanation over relationship advice), and a preference for well-written prompts. Both the coherence and the strength of preferences increase with model capability. Finally, many of the preferences we find (for example, for leisure) are emergent, in the sense of not being explained by training objectives. These results establish an empirical baseline for understanding language model preferences, with implications for alignment and the emerging study of AI welfare.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 42 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

14 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Can Revealed Preferences Clarify LLM Alignment and Steering?8 May
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference7 Mar

Other references · 12

  1. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
  2. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  3. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
  4. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
  5. 2026The Assistant Axis: Situating and Stabilizing the Default Persona of Language Modelscited by 76
  6. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 63
  7. 2025Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?cited by 49
  8. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 41
  9. 2025A Rosetta Stone for AI Benchmarkscited by 29
  10. 2026Knowing But Not Doing: Convergent Morality and Divergent Action in LLMscited by 6
  11. 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
  12. 2025Beyond Mimicry: Preference Coherence in LLMscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards Understanding Sycophancy in Language Models
  2. 2025Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare
  3. 2025Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?
  4. 2026Incoherent Values? Probing LLM Preferences Through Parametric Variation
  5. 2023Can LLMs Capture Human Preferences?
  6. 2026When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models
  7. 2024Improving Context-Aware Preference Modeling for Language Models
  8. 2025Flattery, Fluff, and Fog: Diagnosing and Mitigating Idiosyncratic Biases in Preference Models