Can LLMs make trade-offs involving stipulated pain and pleasure states?
Not on the map. Why?
Abstract (arXiv)
Pleasure and pain play an important role in human decision making by providing a common currency for resolving motivational conflicts. While Large Language Models (LLMs) can generate detailed descriptions of pleasure and pain experiences, it is an open question whether LLMs can recreate the motivational force of pleasure and pain in choice scenarios - a question which may bear on debates about LLM sentience, understood as the capacity for valenced experiential states. We probed this question using a simple game in which the stated goal is to maximise points, but where either the points-maximising option is said to incur a pain penalty or a non-points-maximising option is said to incur a pleasure reward, providing incentives to deviate from points-maximising behaviour. Varying the intensity of the pain penalties and pleasure rewards, we found that Claude 3.5 Sonnet, Command R+, GPT-4o, and GPT-4o mini each demonstrated at least one trade-off in which the majority of responses switched from points-maximisation to pain-minimisation or pleasure-maximisation after a critical threshold of stipulated pain or pleasure intensity is reached. LLaMa 3.1-405b demonstrated some graded sensitivity to stipulated pleasure rewards and pain penalties. Gemini 1.5 Pro and PaLM 2 prioritised pain-avoidance over points-maximisation regardless of intensity, while tending to prioritise points over pleasure regardless of intensity. We discuss the implications of these findings for debates about the possibility of LLM sentience.
Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 38 of its 121 references that are in the map. First posted on arXiv 1 Nov 2024.
Builds on
38 of 121 references.
References · 38
- 2017Attention Is All You Needcited by 59,932
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
- 2023PaLM 2 Technical Reportcited by 1,154
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2023Role-Play with Large Language Modelscited by 290
26 more
- 2022Talking About Large Language Modelscited by 174
- 2022Language models show human-like content effects on reasoning taskscited by 171
- 2021Alignment of Language Agentscited by 167
- 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
- 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
- 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
- 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
- 2024The Ethics of Advanced AI Assistantscited by 103
- 2023Deception Abilities Emerged in Large Language Modelscited by 99
- 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
- 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
- 2023Could a Large Language Model be Conscious?cited by 70
- 2023Turning large language models into cognitive modelscited by 66
- 2024CogBench: a large language model walks into a psychology labcited by 60
- 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 59
- 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
- 2024A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AIcited by 31
- 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
- 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 21
- 2023The feasibility of artificial consciousness through the lens of neurosciencecited by 18
- 2023Revisiting the Reliability of Psychological Scales on Large Language Modelscited by 16
- 2024Simulacra as Conscious Exoticacited by 16
- 2024Does ChatGPT Have a Mind?cited by 13
- 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
- 2024On the attribution of confidence to large language modelscited by 5
- 2023Strengthening the EU AI Act: Defining Key Terms on AI Manipulationcited by 1
Built on by
Citing papers per year: 2025: 9; 2026: 7.
16 papers.
Sort:
By area:
- 2025Large Language Models Report Subjective Experience Under Self-Referential Processingcited by 16
- 2025Agentic Knowledgeable Self-awarenesscited by 8
- 2025Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfarecited by 7
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Modelscited by 3
- 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
- 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanismscited by 3
- 2025Can Large Language Models Develop Gambling Addiction?cited by 2
- 2025Cash or Comfort? How LLMs Value Your Inconveniencecited by 1
- 2025The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systemscited by 1
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questionscited by 1
- 2025Performance Gains of LLMs With Humans in a World of LLMs Versus Humanscited by 0
- 2025Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priorscited by 0
4 more
- 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentalitycited by 0
- 2026How much of a measured AI preference is the model, and how much is the instrument?cited by 0
- 2026The Pain Axis: LLMs Represent Self-Directed Harm and Act on Itcited by 0
- 2026Language Models Act on Hidden Valencecited by 0
- 2026Language Models Act on Hidden Valence28 Sep 2026
- 2026The Pain Axis: LLMs Represent Self-Directed Harm and Act on It14 Sep 2026
- 2026How much of a measured AI preference is the model, and how much is the instrument?24 Aug 2026
- 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms28 May 2026
- 2026Probing Persona-Dependent Preferences in Language Models13 May 2026
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questions7 May 2026
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Models20 Jan 2026
- 2025Large Language Models Report Subjective Experience Under Self-Referential Processing27 Oct 2025
- 2025The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systems30 Sep 2025
- 2025Can Large Language Models Develop Gambling Addiction?26 Sep 2025
- 2025Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare9 Sep 2025
- 2025Cash or Comfort? How LLMs Value Your Inconvenience20 Jun 2025
4 more
- 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality16 Jun 2025
- 2025Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors29 May 2025
- 2025Performance Gains of LLMs With Humans in a World of LLMs Versus Humans13 May 2025
- 2025Agentic Knowledgeable Self-awareness4 Apr 2025
- 2025Large Language Models Report Subjective Experience Under Self-Referential Processingcited by 16
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Modelscited by 3
- 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
- 2025Can Large Language Models Develop Gambling Addiction?cited by 2
- 2025Cash or Comfort? How LLMs Value Your Inconveniencecited by 1
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questionscited by 1
- 2025Performance Gains of LLMs With Humans in a World of LLMs Versus Humanscited by 0
- 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentalitycited by 0
- 2026How much of a measured AI preference is the model, and how much is the instrument?cited by 0
- 2026The Pain Axis: LLMs Represent Self-Directed Harm and Act on Itcited by 0
- 2026Language Models Act on Hidden Valencecited by 0
- 2026Language Models Act on Hidden Valence28 Sep 2026
- 2026The Pain Axis: LLMs Represent Self-Directed Harm and Act on It14 Sep 2026
- 2026How much of a measured AI preference is the model, and how much is the instrument?24 Aug 2026
- 2026Probing Persona-Dependent Preferences in Language Models13 May 2026
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questions7 May 2026
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Models20 Jan 2026
- 2025Large Language Models Report Subjective Experience Under Self-Referential Processing27 Oct 2025
- 2025Can Large Language Models Develop Gambling Addiction?26 Sep 2025
- 2025Cash or Comfort? How LLMs Value Your Inconvenience20 Jun 2025
- 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality16 Jun 2025
- 2025Performance Gains of LLMs With Humans in a World of LLMs Versus Humans13 May 2025