mlsanityData to 7 Oct 2026

Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5

arXiv 2305.04400 ↗First posted 8 May 2023cs.AI cs.CL q-bio.NCcited by 41

Not on the map. Why?

Abstract (arXiv)

A Large Language Model (LLM) is an artificial intelligence system that has been trained on vast amounts of natural language data, enabling it to generate human-like responses to written or spoken language input. GPT-3.5 is an example of an LLM that supports a conversational agent called ChatGPT. In this work, we used a series of novel prompts to determine whether ChatGPT shows heuristics, biases, and other decision effects. We also tested the same prompts on human participants. Across four studies, we found that ChatGPT was influenced by random anchors in making estimates (Anchoring Heuristic, Study 1); it judged the likelihood of two events occurring together to be higher than the likelihood of either event occurring alone, and it was erroneously influenced by salient anecdotal information (Representativeness and Availability Heuristic, Study 2); it found an item to be more efficacious when its features were presented positively rather than negatively - even though both presentations contained identical information (Framing Effect, Study 3); and it valued an owned item more than a newly found item even though the two items were identical (Endowment Effect, Study 4). In each study, human participants showed similar effects. Heuristics and related decision effects in humans are thought to be driven by cognitive and affective processes such as loss aversion and effort reduction. The fact that an LLM - which lacks these processes - also shows such effects invites consideration of the possibility that language may play a role in generating these effects in humans.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 4 of its 37 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

4 of 37 references.

References · 4

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019How Can We Know What Language Models Know?cited by 953
  3. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 244
  4. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 20; 2025: 9; 2026: 7.

41 papers.

Sort:

By area:

  1. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  2. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  3. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  4. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  5. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  6. 2024Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processescited by 38
  7. 2024Cognitive Biases in Large Language Models: A Survey and Mitigation Experimentscited by 20
  8. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  9. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  10. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  11. 2025When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMscited by 14
  12. 2024Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMscited by 12
29 more
  1. 2023Using Artificial Populations to Study Psychological Phenomena in Neural Modelscited by 8
  2. 2024A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptivecited by 8
  3. 2024Do Large Language Models Learn Human-Like Strategic Preferences?cited by 8
  4. 2024Large Language Model Recall Uncertainty is Modulated by the Fan Effectcited by 7
  5. 2025How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulationscited by 7
  6. 2024Can Large Language Models Replace Human Subjects? A Large-Scale Replication of Scenario-Based Experiments in Psychology and Managementcited by 5
  7. 2023Is There Any Social Principle for LLM-Based Agents?cited by 4
  8. 2025The Other Mind: How Language Models Exhibit Human Temporal Cognitioncited by 4
  9. 2024Generative AI-in-the-loop: Integrating LLMs and GPTs into the Next Generation Networkscited by 2
  10. 2024Balancing Rigor and Utility: Mitigating Cognitive Biases in Large Language Models for Multiple-Choice Questionscited by 2
  11. 2024Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Modelscited by 2
  12. 2025The Fragility of Strategic Thinking in Large Language Modelscited by 2
  13. 2023From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Modelscited by 1
  14. 2024Meaningless is better: hashing bias-inducing words in LLM prompts improves performance in logical reasoning and statistical learningcited by 1
  15. 2025Iconicity in Large Language Modelscited by 1
  16. 2026The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoningcited by 1
  17. 2026One Size Fits None: Heuristic Collapse in LLM Investment Advicecited by 1
  18. 202612 Angry AI Agents: Evaluating Multi-Agent LLM Decision-Making Through Cinematic Jury Deliberationcited by 1
  19. 2026Beyond Averages: Evaluating LLMs on Human Survey Replication at the Distributional Levelcited by 1
  20. 2023Applying Standards to Advance Upstream & Downstream Ethics in Large Language Modelscited by 0
  21. 2024Influence of Solution Efficiency and Valence of Instruction on Additive and Subtractive Solution Strategies in Humans and GPT-4cited by 0
  22. 2024Analyzing Large language models chatbots: An experimental approach using a probability testcited by 0
  23. 2025Negation-Induced Forgetting in LLMscited by 0
  24. 2025Replicating Human Social Perception in Generative AI: Evaluating the Valence-Dominance Modelcited by 0
  25. 2025Natural, Artificial, and Human Intelligencescited by 0
  26. 2025Opportunities and Challenges of Natural Language Processing for Low-Resource Senegalese Languages in Social Science Researchcited by 0
  27. 2026DeFrame: Debiasing Large Language Models Against Framing Effectscited by 0
  28. 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detectioncited by 0
  29. 2026Mimicry without understanding: the origins of decision bias in large language modelscited by 0