# Do language models learn typicality judgments from text?

arXiv: https://arxiv.org/abs/2105.02987 · HTML: https://mlsanity.com/paper/2105.02987/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Priming](https://mlsanity.com/paper/2010.03010/) (2020)

## Built on by (most cited first, 30 of 32)

- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning](https://mlsanity.com/paper/2505.17117/) (2025)
- [Inductive reasoning in humans and large language models](https://mlsanity.com/paper/2306.06548/) (2023)
- [Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions](https://mlsanity.com/paper/2205.11658/) (2022)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models](https://mlsanity.com/paper/2210.01963/) (2022)
- [Characterizing the Role of Similarity in the Property Inferences of Language Models](https://mlsanity.com/paper/2410.22590/) (2024)
- [A Property Induction Framework for Neural Language Models](https://mlsanity.com/paper/2205.06910/) (2022)
- [Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow](https://mlsanity.com/paper/2408.08651/) (2024)
- [Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge](https://mlsanity.com/paper/2112.08583/) (2021)
- [Using Artificial Populations to Study Psychological Phenomena in Neural Models](https://mlsanity.com/paper/2308.08032/) (2023)
- [Do Large Language Models Learn Human-Like Strategic Preferences?](https://mlsanity.com/paper/2404.08710/) (2024)
- [How Well Do Deep Learning Models Capture Human Concepts? The Case of the Typicality Effect](https://mlsanity.com/paper/2405.16128/) (2024)
- [Large Language Model Recall Uncertainty is Modulated by the Fan Effect](https://mlsanity.com/paper/2407.06349/) (2024)
- [Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It](https://mlsanity.com/paper/2507.13328/) (2025)
- [Can Language Representation Models Think in Bets?](https://mlsanity.com/paper/2210.07519/) (2022)
- [The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance](https://mlsanity.com/paper/2406.11634/) (2024)
- [Development of Cognitive Intelligence in Pre-trained Language Models](https://mlsanity.com/paper/2407.01047/) (2024)
- [On Semantic Cognition, Inductive Generalization, and Language Models](https://mlsanity.com/paper/2111.02603/) (2021)
- [The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories](https://mlsanity.com/paper/2501.12651/) (2025)
- [Investigating Human-Aligned Large Language Model Uncertainty](https://mlsanity.com/paper/2503.12528/) (2025)
- [On Language Models' Sensitivity to Suspicious Coincidences](https://mlsanity.com/paper/2504.09387/) (2025)
- [Counteracts: Testing Stereotypical Representation in Pre-trained Language Models](https://mlsanity.com/paper/2301.04347/) (2023)
- [Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks](https://mlsanity.com/paper/2302.06942/) (2023)
- [Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT](https://mlsanity.com/paper/2310.13824/) (2023)
- [How Humans and LLMs Organize Conceptual Knowledge: Exploring Subordinate Categories in Italian](https://mlsanity.com/paper/2505.21301/) (2025)
- [Cognitive and Cultural Topology of Linguistic Categories:A Semantic-Pragmatic Metric Approach](https://mlsanity.com/paper/2112.06876/) (2021)
- [LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling](https://mlsanity.com/paper/2308.02903/) (2023)
- [Basic Category Usage in Vision Language Models](https://mlsanity.com/paper/2503.12530/) (2025)

## Abstract (arXiv)

Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language models (LMs) -- informed solely by textual input -- on a prevalent phenomenon in cognitive science: typicality. Inspired by experiments that involve language processing and show robust typicality effects in humans, we propose two tests for LMs. Our first test targets whether typicality modulates LM probabilities in assigning taxonomic category memberships to items. The second test investigates sensitivities to typicality in LMs' probabilities when extending new information about items to their categories. Both tests show modest -- but not completely absent -- correspondence between LMs and humans, suggesting that text-based exposure alone is insufficient to acquire typicality knowledge.