mlsanityData to 14 Oct 2026

Large Language Models Assume People are More Rational than We Really are

arXiv 2406.17055 ↗First posted 24 Jun 2024cs.CL cs.AI cs.CYcited by 54

Not on the map. Why?

Abstract (arXiv)

In order for AI systems to communicate effectively with people, they must understand how we make decisions. However, people's decisions are not always rational, so the implicit internal models of human decision-making in Large Language Models (LLMs) must account for this. Previous empirical evidence seems to suggest that these implicit models are accurate -- LLMs offer believable proxies of human behavior, acting how we expect humans would in everyday interactions. However, by comparing LLM behavior and predictions to a large dataset of human decisions, we find that this is actually not the case: when both simulating and predicting people's choices, a suite of cutting-edge LLMs (GPT-4o & 4-Turbo, Llama-3-8B & 70B, Claude 3 Opus) assume that people are more rational than we really are. Specifically, these models deviate from human behavior and align more closely with a classic model of rational choice -- expected value theory. Interestingly, people also tend to assume that other people are rational when interpreting their behavior. As a consequence, when we compare the inferences that LLMs and people draw from the decisions of others using another psychological dataset, we find that these inferences are highly correlated. Thus, the implicit decision-making models of LLMs appear to be aligned with the human expectation that other people will act rationally, rather than with how people actually act.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 45 of its 80 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

45 of 80 references.

References · 45

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2023LIMA: Less Is More for Alignmentcited by 1,100
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  12. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
33 more
  1. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  2. 2022Using cognitive psychology to understand GPT-3cited by 398
  3. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  4. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  5. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  6. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  7. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  8. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
  9. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  10. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  11. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  12. 2022Human heuristics for AI-generated language are flawedcited by 105
  13. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  14. 2023Turning large language models into cognitive modelscited by 67
  15. 2024CogBench: a large language model walks into a psychology labcited by 63
  16. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  17. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 55
  18. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 49
  19. 2015Learning the Preferences of Ignorant, Inconsistent Agentscited by 45
  20. 2024A Survey on Large Language Model Hallucination via a Creativity Perspectivecited by 38
  21. 2023Does GPT-4 pass the Turing test?cited by 37
  22. 2023Language Models Trained on Media Diets Can Predict Public Opinioncited by 30
  23. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  24. 2024How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?cited by 27
  25. 2019Cognitive Model Priors for Predicting Human Decisionscited by 25
  26. 2021Cognitive science as a source of forward and inverse models of human decisions for robotics and controlcited by 22
  27. 2021Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mindcited by 19
  28. 2023Inferring the Goals of Communicating Agents from Actions and Instructionscited by 17
  29. 2019Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learningcited by 15
  30. 2023Improving Interpersonal Communication by Simulating Audiences with Language Modelscited by 14
  31. 2024LLM-driven Imitation of Subrational Behavior : Illusion or Reality?cited by 12
  32. 2024IMBUE: Improving Interpersonal Effectiveness through Simulation and Just-in-time Feedback with Human-Language Model Interactioncited by 12
  33. 2023Social Contract AI: Aligning AI Assistants with Implicit Group Normscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 24; 2026: 25.

54 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  2. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 109
  3. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 57
  4. 2025Base Models Beat Aligned Models at Randomness and Creativitycited by 51
  5. 2025Bayesian Teaching Enables Probabilistic Reasoning in Large Language Modelscited by 37
  6. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 28
  7. 2025On Benchmarking Human-Like Intelligence in Machinescited by 25
  8. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  9. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  10. 2026FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insightscited by 16
  11. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 14
  12. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
38 more
  1. 2025A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experimentscited by 12
  2. 2025Levels of Analysis for Large Language Modelscited by 10
  3. 2025MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automationcited by 10
  4. 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMscited by 10
  5. 2024The Double-Edged Sword of Behavioral Responses in Strategic Classification: Theory and User Studiescited by 9
  6. 2025Distributive Fairness in Large Language Models: Evaluating Alignment with Human Valuescited by 9
  7. 2026Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Modelscited by 9
  8. 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 8
  9. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  10. 2025Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representationscited by 6
  11. 2024Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-makingcited by 5
  12. 2025LLM Agents Are Hypersensitive to Nudgescited by 5
  13. 2026Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilitiescited by 5
  14. 2026Can Revealed Preferences Clarify LLM Alignment and Steering?cited by 5
  15. 2025Evaluating Language Models' Evaluations of Gamescited by 4
  16. 2026Alignment Makes Language Models Normative, Not Descriptivecited by 4
  17. 2025LLM Agents Display Human Biases but Exhibit Distinct Learning Patternscited by 3
  18. 2026Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choicescited by 3
  19. 2026MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Modelscited by 3
  20. 2025Revisiting Rogers' Paradox in the Context of Human-AI Interactioncited by 2
  21. 2025The Emergence of Altruism in Large-Language-Model Agents Societycited by 2
  22. 2025Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?cited by 2
  23. 2026Sentipolis: Emotion-Aware Agents for Social Simulationscited by 2
  24. 2026Using Cognitive Models to Improve Language Model Simulation of Human Persuasion Gamescited by 2
  25. 2025Predicting Human Choice Between Textually Described Lotteriescited by 1
  26. 2025Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?cited by 1
  27. 2025Cash or Comfort? How LLMs Value Your Inconveniencecited by 1
  28. 2026Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulationcited by 1
  29. 2026Position: Behavioral Systems Require Behavioral Testscited by 1
  30. 2026Predicting Biased Human Decision-Making with Large Language Models in Conversational Settingscited by 0
  31. 2026Eliciting Trustworthiness Priors of Large Language Models via Economic Gamescited by 0
  32. 2026Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agentscited by 0
  33. 2026LLMs Should Not Yet Be Credited with Decision Explanationcited by 0
  34. 2026The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Modelscited by 0
  35. 2026LLM Consumer Behavior Theory: Foundations of a Novel Research Fieldcited by 0
  36. 2026Will Scaling Improve Social Simulation with LLMs?cited by 0
  37. 2026Before the Arrest: Benchmarking LLMs on Criminal Profiling from Incomplete Evidencecited by 0
  38. 2026Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allowcited by 0

50 most cited of 54 All 54 (JSON)