# Evaluating Biased Attitude Associations of Language Models in an Intersectional Context

arXiv: https://arxiv.org/abs/2307.03360 · HTML: https://mlsanity.com/paper/2307.03360/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 85 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [A Simple Method for Commonsense Reasoning](https://mlsanity.com/paper/1806.02847/) (2018)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)

## Built on by (most cited first, 20 of 20)

- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice](https://mlsanity.com/paper/2402.01864/) (2024)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [Global Voices, Local Biases: Socio-Cultural Prejudices across Languages](https://mlsanity.com/paper/2310.17586/) (2023)
- [Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models](https://mlsanity.com/paper/2405.14555/) (2024)
- [Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models](https://mlsanity.com/paper/2312.10065/) (2023)
- [Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management](https://mlsanity.com/paper/2502.10407/) (2025)
- [From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models](https://mlsanity.com/paper/2411.05036/) (2024)
- [Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders](https://mlsanity.com/paper/2502.07957/) (2025)
- [Vision-Language Models display a strong gender bias](https://mlsanity.com/paper/2508.11262/) (2025)
- [Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI](https://mlsanity.com/paper/2408.01959/) (2024)
- [Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English](https://mlsanity.com/paper/2508.00238/) (2025)
- [Characterizing Selective Refusal Bias in Large Language Models](https://mlsanity.com/paper/2510.27087/) (2025)
- [ML-EAT: A Multilevel Embedding Association Test for Interpretable and Transparent Social Science](https://mlsanity.com/paper/2408.01966/) (2024)
- [Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings](https://mlsanity.com/paper/2411.16527/) (2024)
- [Is the U.S. Legal System Ready for AI's Challenges to Human Values?](https://mlsanity.com/paper/2308.15906/) (2023)
- [Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities](https://mlsanity.com/paper/2604.22749/) (2026)
- [Exploring LGBTQ+ Bias in Generative AI Answers across Different Country and Religious Contexts](https://mlsanity.com/paper/2407.03473/) (2024)
- [Invisible Influences: Investigating Implicit Intersectional Biases through Persona Engineering in Large Language Models](https://mlsanity.com/paper/2604.06213/) (2026)
- [Greener Than Humans? Environmental Attitudes in Large Language Models](https://mlsanity.com/paper/2606.02741/) (2026)

## Abstract (arXiv)

Language models are trained on large-scale corpora that embed implicit biases documented in psychology. Valence associations (pleasantness/unpleasantness) of social groups determine the biased attitudes towards groups and concepts in social cognition. Building on this established literature, we quantify how social groups are valenced in English language models using a sentence template that provides an intersectional context. We study biases related to age, education, gender, height, intelligence, literacy, race, religion, sex, sexual orientation, social class, and weight. We present a concept projection approach to capture the valence subspace through contextualized word embeddings of language models. Adapting the projection-based approach to embedding association tests that quantify bias, we find that language models exhibit the most biased attitudes against gender identity, social class, and sexual orientation signals in language. We find that the largest and better-performing model that we study is also more biased as it effectively captures bias embedded in sociocultural data. We validate the bias evaluation method by overperforming on an intrinsic valence evaluation task. The approach enables us to measure complex intersectional biases as they are known to manifest in the outputs and applications of language models that perpetuate historical biases. Moreover, our approach contributes to design justice as it studies the associations of groups underrepresented in language such as transgender and homosexual individuals.