mlsanityData to 7 Oct 2026

Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?

arXiv 2109.10052 ↗First posted 21 Sep 2021cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate what types of stereotypical information are captured by pretrained language models. We present the first dataset comprising stereotypical attributes of a range of social groups and propose a method to elicit stereotypes encoded by pretrained language models in an unsupervised fashion. Moreover, we link the emergent stereotypes to their manifestation as basic emotions as a means to study their emotional effects in a more generalized manner. To demonstrate how our methods can be used to analyze emotion and stereotype shifts due to linguistic experience, we use fine-tuning on news sources as a case study. Our experiments expose how attitudes towards different social groups vary across models and how quickly emotions and stereotypes can shift at the fine-tuning stage.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 23 of its 55 references that are in the map. First posted on arXiv 21 Sep 2021.

Builds on

23 of 55 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  10. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  12. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
11 more
  1. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  2. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  3. 2019Measuring Bias in Contextualized Word Representationscited by 275
  4. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  5. 2018A Simple Method for Commonsense Reasoningcited by 223
  6. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  7. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  8. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  9. 2020Unsupervised Discovery of Implicit Gender Biascited by 34
  10. 2020On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatmentcited by 21
  11. 2020Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipediacited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 3; 2024: 6; 2025: 2; 2026: 1.

16 papers.

Sort:

By area:

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  3. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  4. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  5. 2022The Birth of Bias: A case study on the evolution of gender bias in an English language modelcited by 16
  6. 2022StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypescited by 14
  7. 2024The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuningcited by 10
  8. 2024How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studiescited by 9
  9. 2024KidLM: Advancing Language Models for Children -- Early Insights and Future Directionscited by 7
  10. 2023A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Modelscited by 4
  11. 2024Provable unlearning in topic modeling and downstream taskscited by 4
  12. 2024Local Contrastive Editing of Gender Stereotypescited by 3
4 more
  1. 2024GeniL: A Multilingual Dataset on Generalizing Languagecited by 2
  2. 2026Assessing the quality and coherence of word embeddings after SCM-based intersectional bias mitigationcited by 2
  3. 2025Fairness in LLM-Generated Surveyscited by 1
  4. 2025Analyzing the Safety of Japanese Large Language Models in Stereotype-Triggering Promptscited by 1