mlsanityData to 7 Oct 2026

Can LLMs make trade-offs involving stipulated pain and pleasure states?

arXiv 2411.02432 ↗First posted 1 Nov 2024cs.CL cs.AI cs.CYcited by 16

Not on the map. Why?

Abstract (arXiv)

Pleasure and pain play an important role in human decision making by providing a common currency for resolving motivational conflicts. While Large Language Models (LLMs) can generate detailed descriptions of pleasure and pain experiences, it is an open question whether LLMs can recreate the motivational force of pleasure and pain in choice scenarios - a question which may bear on debates about LLM sentience, understood as the capacity for valenced experiential states. We probed this question using a simple game in which the stated goal is to maximise points, but where either the points-maximising option is said to incur a pain penalty or a non-points-maximising option is said to incur a pleasure reward, providing incentives to deviate from points-maximising behaviour. Varying the intensity of the pain penalties and pleasure rewards, we found that Claude 3.5 Sonnet, Command R+, GPT-4o, and GPT-4o mini each demonstrated at least one trade-off in which the majority of responses switched from points-maximisation to pain-minimisation or pleasure-maximisation after a critical threshold of stipulated pain or pleasure intensity is reached. LLaMa 3.1-405b demonstrated some graded sensitivity to stipulated pleasure rewards and pain penalties. Gemini 1.5 Pro and PaLM 2 prioritised pain-avoidance over points-maximisation regardless of intensity, while tending to prioritise points over pleasure regardless of intensity. We discuss the implications of these findings for debates about the possibility of LLM sentience.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 38 of its 121 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

38 of 121 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2023A Comprehensive Overview of Large Language Modelscited by 639
  12. 2023Role-Play with Large Language Modelscited by 290
26 more
  1. 2022Talking About Large Language Modelscited by 174
  2. 2022Language models show human-like content effects on reasoning taskscited by 171
  3. 2021Alignment of Language Agentscited by 167
  4. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  5. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  6. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  7. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  8. 2024The Ethics of Advanced AI Assistantscited by 103
  9. 2023Deception Abilities Emerged in Large Language Modelscited by 99
  10. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
  11. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
  12. 2023Could a Large Language Model be Conscious?cited by 70
  13. 2023Turning large language models into cognitive modelscited by 66
  14. 2024CogBench: a large language model walks into a psychology labcited by 60
  15. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 59
  16. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  17. 2024A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AIcited by 31
  18. 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
  19. 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 21
  20. 2023The feasibility of artificial consciousness through the lens of neurosciencecited by 18
  21. 2023Revisiting the Reliability of Psychological Scales on Large Language Modelscited by 16
  22. 2024Simulacra as Conscious Exoticacited by 16
  23. 2024Does ChatGPT Have a Mind?cited by 13
  24. 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
  25. 2024On the attribution of confidence to large language modelscited by 5
  26. 2023Strengthening the EU AI Act: Defining Key Terms on AI Manipulationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 7.

16 papers.

Sort:

By area: