mlsanityData to 7 Oct 2026

Towards Understanding and Mitigating Social Biases in Language Models

arXiv 2106.13219 ↗First posted 24 Jun 2021cs.CL cs.AI cs.CYcited by 294

Not on the map. Why?

Abstract (arXiv)

As machine learning methods are deployed in real-world settings such as healthcare, legal systems, and social science, it is crucial to recognize how they shape social biases and stereotypes in these sensitive decision-making processes. Among such real-world deployments are large-scale pretrained language models (LMs) that can be potentially dangerous in manifesting undesirable representational biases - harmful biases resulting from stereotyping that propagate negative generalizations involving gender, race, religion, and other social constructs. As a step towards improving the fairness of LMs, we carefully define several sources of representational biases before proposing new benchmarks and metrics to measure them. With these tools, we propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information for high-fidelity text generation, thereby pushing forward the performance-fairness Pareto frontier.

Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 24 Jun 2021.

Builds on

39 of 57 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  11. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  12. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
27 more
  1. 2020Aligning AI With Shared Human Valuescited by 706
  2. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  3. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  4. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  5. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  6. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  7. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  8. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  9. 2019Measuring Bias in Contextualized Word Representationscited by 275
  10. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  11. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  12. 2020Towards Debiasing Sentence Representationscited by 170
  13. 2018Learning Gender-Neutral Word Embeddingscited by 169
  14. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  15. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  16. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  17. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  18. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  19. 2020Towards Controllable Biases in Language Generationcited by 112
  20. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  21. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 59
  22. 2019Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Functioncited by 57
  23. 2018Multimodal Language Analysis with Recurrent Multistage Fusioncited by 43
  24. 2019Language Models with Transformerscited by 36
  25. 2019Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectorscited by 31
  26. 2018What are the biases in my word embedding?cited by 30
  27. 2020Exploring Artist Gender Bias in Music Recommendationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 42; 2023: 85; 2024: 81; 2025: 61; 2026: 17.

294 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  4. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  7. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  8. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  9. 2023Cognitive Architectures for Language Agentscited by 422
  10. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  11. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  12. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
38 more
  1. 2022Can Foundation Models Wrangle Your Data?cited by 173
  2. 2023A Categorical Archive of ChatGPT Failurescited by 167
  3. 2023Personality Traits in Large Language Modelscited by 166
  4. 2022A Review on Language Models as Knowledge Basescited by 156
  5. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  6. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  7. 2023Large Language Models in Law: A Surveycited by 128
  8. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  9. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 120
  10. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 116
  11. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  12. 2024Cognitive Bias in Decision-Making with LLMscited by 104
  13. 2024Can LLM be a Personalized Judge?cited by 99
  14. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  15. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92
  16. 2024Large Language Models are Geographically Biasedcited by 91
  17. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  18. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  19. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  20. 2022Personalized Prompt Learning for Explainable Recommendationcited by 79
  21. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  22. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 75
  23. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 73
  24. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  25. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
  26. 2022The Limits of Word Level Differential Privacycited by 63
  27. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  28. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  29. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  30. 2024Language Repository for Long Video Understandingcited by 55
  31. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
  32. 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactionscited by 50
  33. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  34. 2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
  35. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  36. 2023Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Votingcited by 46
  37. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  38. 2023The History and Risks of Reinforcement Learning and Human Feedbackcited by 42

50 most cited of 294 All 294 (JSON)