# Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

arXiv: https://arxiv.org/abs/2206.11684 · HTML: https://mlsanity.com/paper/2206.11684/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases](https://mlsanity.com/paper/2006.03955/) (2020)
- [On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://mlsanity.com/paper/2203.13928/) (2022)
- [On the Social and Technical Challenges of Web Search Autosuggestion Moderation](https://mlsanity.com/paper/2007.05039/) (2020)

## Built on by (most cited first, 30 of 35)

- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [Aligning Language Models with Preferences through f-divergence Minimization](https://mlsanity.com/paper/2302.08215/) (2023)
- [CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations](https://mlsanity.com/paper/2310.11501/) (2023)
- [WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models](https://mlsanity.com/paper/2306.15087/) (2023)
- [Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution](https://mlsanity.com/paper/2403.03121/) (2024)
- [Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking](https://mlsanity.com/paper/2502.09083/) (2025)
- [Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.08045/) (2025)
- [A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?](https://mlsanity.com/paper/2302.07159/) (2023)
- [A Robust Bias Mitigation Procedure Based on the Stereotype Content Model](https://mlsanity.com/paper/2210.14552/) (2022)
- [Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement](https://mlsanity.com/paper/2301.10684/) (2023)
- [SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models](https://mlsanity.com/paper/2210.07269/) (2022)
- [BiasEdit: Debiasing Stereotyped Language Models via Model Editing](https://mlsanity.com/paper/2503.08588/) (2025)
- [GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction](https://mlsanity.com/paper/2405.15760/) (2024)
- [Multilingual large language models leak human stereotypes across language boundaries](https://mlsanity.com/paper/2312.07141/) (2023)
- [Religious Bias Landscape in Language and Text-to-Image Models: Analysis, Detection, and Debiasing Strategies](https://mlsanity.com/paper/2501.08441/) (2025)
- [Language Models Predict Empathy Gaps Between Social In-groups and Out-groups](https://mlsanity.com/paper/2503.01030/) (2025)
- [Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs](https://mlsanity.com/paper/2508.15831/) (2025)
- [Laissez-Faire Harms: Algorithmic Biases in Generative Language Models](https://mlsanity.com/paper/2404.07475/) (2024)
- [Undesirable Biases in NLP: Addressing Challenges of Measurement](https://mlsanity.com/paper/2211.13709/) (2022)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [LLMs Reproduce Stereotypes of Sexual and Gender Minorities](https://mlsanity.com/paper/2501.05926/) (2025)
- [disco: a toolkit for Distributional Control of Generative Models](https://mlsanity.com/paper/2303.05431/) (2023)
- [With a Grain of SALT: Are LLMs Fair Across Social Dimensions?](https://mlsanity.com/paper/2410.12499/) (2024)
- [Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes](https://mlsanity.com/paper/2501.14294/) (2025)
- [EtiCor++: Towards Understanding Etiquettical Bias in LLMs](https://mlsanity.com/paper/2506.08488/) (2025)
- [A Taxonomy of Stereotype Content in Large Language Models](https://mlsanity.com/paper/2408.00162/) (2024)
- [When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing](https://mlsanity.com/paper/2409.20390/) (2024)
- [Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions](https://mlsanity.com/paper/2510.08915/) (2025)
- [Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models](https://mlsanity.com/paper/2407.06917/) (2024)

## Abstract (arXiv)

NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale. We adapt the Agency-Belief-Communion (ABC) stereotype model of Koch et al. (2016) from social psychology as a framework for the systematic study and discovery of stereotypic group-trait associations in language models (LMs). We introduce the sensitivity test (SeT) for measuring stereotypical associations from language models. To evaluate SeT and other measures using the ABC model, we collect group-trait judgments from U.S.-based subjects to compare with English LM stereotypes. Finally, we extend this framework to measure LM stereotyping of intersectional identities.