# Can LLMs make trade-offs involving stipulated pain and pleasure states?

arXiv: https://arxiv.org/abs/2411.02432 · HTML: https://mlsanity.com/paper/2411.02432/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 121 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Evaluating Frontier Models for Dangerous Capabilities](https://mlsanity.com/paper/2403.13793/) (2024)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Consciousness in Artificial Intelligence: Insights from the Science of Consciousness](https://mlsanity.com/paper/2308.08708/) (2023)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [Evaluating Language-Model Agents on Realistic Autonomous Tasks](https://mlsanity.com/paper/2312.11671/) (2023)
- [Could a Large Language Model be Conscious?](https://mlsanity.com/paper/2303.07103/) (2023)
- [Turning large language models into cognitive models](https://mlsanity.com/paper/2306.03917/) (2023)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [LLMs achieve adult human performance on higher-order theory of mind tasks](https://mlsanity.com/paper/2405.18870/) (2024)
- [Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench](https://mlsanity.com/paper/2308.03656/) (2023)
- [A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI](https://mlsanity.com/paper/2404.15058/) (2024)
- [Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench](https://mlsanity.com/paper/2310.01386/) (2023)

## Built on by (most cited first, 16 of 16)

- [Large Language Models Report Subjective Experience Under Self-Referential Processing](https://mlsanity.com/paper/2510.24797/) (2025)
- [Agentic Knowledgeable Self-awareness](https://mlsanity.com/paper/2504.03553/) (2025)
- [Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare](https://mlsanity.com/paper/2509.07961/) (2025)
- [Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms](https://mlsanity.com/paper/2605.30169/) (2026)
- [No Reliable Evidence of Self-Reported Sentience in Small Large Language Models](https://mlsanity.com/paper/2601.15334/) (2026)
- [Probing Persona-Dependent Preferences in Language Models](https://mlsanity.com/paper/2605.13339/) (2026)
- [Can Large Language Models Develop Gambling Addiction?](https://mlsanity.com/paper/2509.22818/) (2025)
- [Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality](https://mlsanity.com/paper/2506.13403/) (2025)
- [Cash or Comfort? How LLMs Value Your Inconvenience](https://mlsanity.com/paper/2506.17367/) (2025)
- [The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systems](https://mlsanity.com/paper/2509.26126/) (2025)
- [AI and Consciousness: Shifting Focus Towards Tractable Questions](https://mlsanity.com/paper/2605.06965/) (2026)
- [The Pain Axis: LLMs Represent Self-Directed Harm and Act on It](https://mlsanity.com/paper/2609.16247/) (2026)
- [Performance Gains of LLMs With Humans in a World of LLMs Versus Humans](https://mlsanity.com/paper/2505.08902/) (2025)
- [Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors](https://mlsanity.com/paper/2505.23323/) (2025)
- [How much of a measured AI preference is the model, and how much is the instrument?](https://mlsanity.com/paper/2608.23641/) (2026)
- [Language Models Act on Hidden Valence](https://mlsanity.com/paper/2609.35591/) (2026)

## Abstract (arXiv)

Pleasure and pain play an important role in human decision making by providing a common currency for resolving motivational conflicts. While Large Language Models (LLMs) can generate detailed descriptions of pleasure and pain experiences, it is an open question whether LLMs can recreate the motivational force of pleasure and pain in choice scenarios - a question which may bear on debates about LLM sentience, understood as the capacity for valenced experiential states. We probed this question using a simple game in which the stated goal is to maximise points, but where either the points-maximising option is said to incur a pain penalty or a non-points-maximising option is said to incur a pleasure reward, providing incentives to deviate from points-maximising behaviour. Varying the intensity of the pain penalties and pleasure rewards, we found that Claude 3.5 Sonnet, Command R+, GPT-4o, and GPT-4o mini each demonstrated at least one trade-off in which the majority of responses switched from points-maximisation to pain-minimisation or pleasure-maximisation after a critical threshold of stipulated pain or pleasure intensity is reached. LLaMa 3.1-405b demonstrated some graded sensitivity to stipulated pleasure rewards and pain penalties. Gemini 1.5 Pro and PaLM 2 prioritised pain-avoidance over points-maximisation regardless of intensity, while tending to prioritise points over pleasure regardless of intensity. We discuss the implications of these findings for debates about the possibility of LLM sentience.