# Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMs

arXiv: https://arxiv.org/abs/2404.01461 · HTML: https://mlsanity.com/paper/2404.01461/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Towards Debiasing Sentence Representations](https://mlsanity.com/paper/2007.08100/) (2020)
- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Mitigating Political Bias in Language Models Through Reinforced Calibration](https://mlsanity.com/paper/2104.14795/) (2021)
- [What's in a Name? Reducing Bias in Bios without Access to Protected Attributes](https://mlsanity.com/paper/1904.05233/) (2019)
- [Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks](https://mlsanity.com/paper/2306.05550/) (2023)
- [Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5](https://mlsanity.com/paper/2305.04400/) (2023)

## Built on by (most cited first, 13 of 13)

- [RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph](https://mlsanity.com/paper/2410.14684/) (2024)
- [A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners](https://mlsanity.com/paper/2406.11050/) (2024)
- [Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processes](https://mlsanity.com/paper/2404.17218/) (2024)
- [Towards Rationality in Language and Multimodal Agents: A Survey](https://mlsanity.com/paper/2406.00252/) (2024)
- [CBEval: A framework for evaluating and interpreting cognitive biases in LLMs](https://mlsanity.com/paper/2412.03605/) (2024)
- [The Bias is in the Details: An Assessment of Cognitive Bias in LLMs](https://mlsanity.com/paper/2509.22856/) (2025)
- [Balancing Rigor and Utility: Mitigating Cognitive Biases in Large Language Models for Multiple-Choice Questions](https://mlsanity.com/paper/2406.10999/) (2024)
- [From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition](https://mlsanity.com/paper/2410.13259/) (2024)
- [SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers](https://mlsanity.com/paper/2602.05115/) (2026)
- [Meaningless is better: hashing bias-inducing words in LLM prompts improves performance in logical reasoning and statistical learning](https://mlsanity.com/paper/2411.17304/) (2024)
- [The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning](https://mlsanity.com/paper/2603.29025/) (2026)
- [Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets](https://mlsanity.com/paper/2609.29509/) (2026)
- [CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models](https://mlsanity.com/paper/2604.01366/) (2026)

## Abstract (arXiv)

Although large language models (LLMs) have demonstrated remarkable proficiency in modeling text and generating human-like text, they may exhibit biases acquired from training data in doing so. Specifically, LLMs may be susceptible to a common cognitive trap in human decision-making called the representativeness heuristic. This is a concept in psychology that refers to judging the likelihood of an event based on how closely it resembles a well-known prototype or typical example, versus considering broader facts or statistical evidence. This research investigates the impact of the representativeness heuristic on LLM reasoning. We created ReHeAT (Representativeness Heuristic AI Testing), a dataset containing a series of problems spanning six common types of representativeness heuristics. Experiments reveal that four LLMs applied to ReHeAT all exhibited representativeness heuristic biases. We further identify that the model's reasoning steps are often incorrectly based on a stereotype rather than on the problem's description. Interestingly, the performance improves when adding a hint in the prompt to remind the model to use its knowledge. This suggests the uniqueness of the representativeness heuristic compared to traditional biases. It can occur even when LLMs possess the correct knowledge while falling into a cognitive trap. This highlights the importance of future research focusing on the representativeness heuristic in model reasoning and decision-making and on developing solutions to address it.