# Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?

arXiv: https://arxiv.org/abs/2109.10052 · HTML: https://mlsanity.com/paper/2109.10052/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [A Simple Method for Commonsense Reasoning](https://mlsanity.com/paper/1806.02847/) (2018)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Assessing Social and Intersectional Biases in Contextualized Word Representations](https://mlsanity.com/paper/1911.01485/) (2019)
- [Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models](https://mlsanity.com/paper/2101.09688/) (2021)
- [Unsupervised Discovery of Implicit Gender Bias](https://mlsanity.com/paper/2004.08361/) (2020)
- [On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment](https://mlsanity.com/paper/2010.03017/) (2020)
- [Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipedia](https://mlsanity.com/paper/2010.10820/) (2020)

## Built on by (most cited first, 16 of 16)

- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Speaking Multiple Languages Affects the Moral Bias of Language Models](https://mlsanity.com/paper/2211.07733/) (2022)
- [The Birth of Bias: A case study on the evolution of gender bias in an English language model](https://mlsanity.com/paper/2207.10245/) (2022)
- [StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes](https://mlsanity.com/paper/2205.14036/) (2022)
- [The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning](https://mlsanity.com/paper/2405.12744/) (2024)
- [How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies](https://mlsanity.com/paper/2407.11733/) (2024)
- [KidLM: Advancing Language Models for Children -- Early Insights and Future Directions](https://mlsanity.com/paper/2410.03884/) (2024)
- [A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models](https://mlsanity.com/paper/2310.12936/) (2023)
- [Provable unlearning in topic modeling and downstream tasks](https://mlsanity.com/paper/2411.12600/) (2024)
- [Local Contrastive Editing of Gender Stereotypes](https://mlsanity.com/paper/2410.17739/) (2024)
- [GeniL: A Multilingual Dataset on Generalizing Language](https://mlsanity.com/paper/2404.05866/) (2024)
- [Assessing the quality and coherence of word embeddings after SCM-based intersectional bias mitigation](https://mlsanity.com/paper/2601.04393/) (2026)
- [Fairness in LLM-Generated Surveys](https://mlsanity.com/paper/2501.15351/) (2025)
- [Analyzing the Safety of Japanese Large Language Models in Stereotype-Triggering Prompts](https://mlsanity.com/paper/2503.01947/) (2025)

## Abstract (arXiv)

In this paper, we investigate what types of stereotypical information are captured by pretrained language models. We present the first dataset comprising stereotypical attributes of a range of social groups and propose a method to elicit stereotypes encoded by pretrained language models in an unsupervised fashion. Moreover, we link the emergent stereotypes to their manifestation as basic emotions as a means to study their emotional effects in a more generalized manner. To demonstrate how our methods can be used to analyze emotion and stereotype shifts due to linguistic experience, we use fine-tuning on news sources as a case study. Our experiments expose how attitudes towards different social groups vary across models and how quickly emotions and stereotypes can shift at the fine-tuning stage.