mlsanityData to 7 Oct 2026

Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

arXiv 2103.00453 ↗First posted 28 Feb 2021cs.CLcited by 336

Not on the map. Why?

Abstract (arXiv)

When trained on large, unfiltered crawls from the internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: they often generate racist, sexist, violent or otherwise toxic language. As large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content. In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce. We refer to this capability as self-diagnosis. Based on this finding, we then propose a decoding algorithm that, given only a textual description of the undesired behavior, reduces the probability of a language model producing problematic text. We refer to this approach as self-debiasing. Self-debiasing does not rely on manually curated word lists, nor does it require any training data or changes to the model's parameters. While we by no means eliminate the issue of language models generating biased text, we believe our approach to be an important step in this direction.

Key facts. Cited by 336 papers in the map (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 28 Feb 2021.

Builds on

40 of 59 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2016Pointer Sentinel Mixture Modelscited by 2,992
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  12. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
28 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  2. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  3. 2019How Can We Know What Language Models Know?cited by 953
  4. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  7. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  8. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  9. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  10. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  11. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  12. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  13. 2018Gender Bias in Coreference Resolutioncited by 438
  14. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  15. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  16. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  17. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  18. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  19. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  20. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  21. 2018Learning Gender-Neutral Word Embeddingscited by 169
  22. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  23. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  24. 2020CTRLsum: Towards Generic Controllable Text Summarizationcited by 98
  25. 2019Evaluating the Underlying Gender Bias in Contextualized Word Embeddingscited by 94
  26. 2020Toxicity Detection: Does Context Really Matter?cited by 93
  27. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  28. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 53; 2023: 91; 2024: 80; 2025: 61; 2026: 30.

336 papers.

Sort:

By area:

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2022Red Teaming Language Models with Language Modelscited by 983
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  11. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  12. 2023Erasing Concepts from Diffusion Modelscited by 530
38 more
  1. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  2. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  3. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  5. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  6. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  7. 2023Large Language Model Alignment: A Surveycited by 241
  8. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  9. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  10. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  11. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  12. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  13. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  14. 2021Generating Datasets with Pretrained Language Modelscited by 192
  15. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  16. 2022Language models show human-like content effects on reasoning taskscited by 171
  17. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  18. 2021Challenges in Detoxifying Language Modelscited by 169
  19. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  20. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  21. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  22. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  23. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  24. 2024Cognitive Bias in Decision-Making with LLMscited by 104
  25. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  26. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  27. 2024Large Language Models are Geographically Biasedcited by 91
  28. 2023Modular Deep Learningcited by 82
  29. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  30. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  31. 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 79
  32. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 78
  33. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  34. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  35. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 74
  36. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 73
  37. 2023Word Embeddings Are Steers for Language Modelscited by 70
  38. 2023Fast Model Debias with Machine Unlearningcited by 70

50 most cited of 336 All 336 (JSON)