mlsanityData to 7 Oct 2026

BiasEdit: Debiasing Stereotyped Language Models via Model Editing

arXiv 2503.08588 ↗First posted 11 Mar 2025cs.CL cs.AI cs.CYcited by 18

Not on the map. Why?

Abstract (arXiv)

Previous studies have established that language models manifest stereotyped biases. Existing debiasing strategies, such as retraining a model with counterfactual data, representation projection, and prompting often fail to efficiently eliminate bias or directly alter the models' biased internal representations. To address these issues, we propose BiasEdit, an efficient model editing method to remove stereotypical bias from language models through lightweight networks that act as editors to generate parameter updates. BiasEdit employs a debiasing loss guiding editor networks to conduct local edits on partial parameters of a language model for debiasing while preserving the language modeling abilities during editing through a retention loss. Experiments on StereoSet and Crows-Pairs demonstrate the effectiveness, efficiency, and robustness of BiasEdit in eliminating bias compared to tangental debiasing baselines and little to no impact on the language models' general capabilities. In addition, we conduct bias tracing to probe bias in various modules and explore bias editing impacts on different components of language models.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 55 of its 86 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

55 of 86 references.

References · 55

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2022Mass-Editing Memory in a Transformercited by 976
  8. 2019How Can We Know What Language Models Know?cited by 953
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  12. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
38 more
  1. 2021Editing Factual Knowledge in Language Modelscited by 548
  2. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  3. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  4. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  5. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  6. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  7. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  8. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  9. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  10. 2020Modifying Memories in Transformer Modelscited by 222
  11. 2018Gender Bias in Neural Natural Language Processingcited by 210
  12. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  13. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  14. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  15. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  16. 2020Towards Debiasing Sentence Representationscited by 170
  17. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  18. 2020Editable Neural Networkscited by 154
  19. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  20. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  21. 2020Towards Controllable Biases in Language Generationcited by 112
  22. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  23. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 92
  24. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 74
  25. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  26. 2023Nationality Bias in Text Generationcited by 50
  27. 2022How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysiscited by 42
  28. 2022Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Modelscited by 35
  29. 2023Evaluating the Reversal Curse in Model Editingcited by 34
  30. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  31. 2023On Evaluating and Mitigating Gender Biases in Multilingual Settingscited by 31
  32. 2022Geographic and Geopolitical Biases of Language Modelscited by 29
  33. 2023DocTER: Evaluating Document-based Knowledge Editingcited by 26
  34. 2023Parameter-efficient Modularised Bias Mitigation via AdapterFusioncited by 25
  35. 2023Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projectioncited by 25
  36. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  37. 2022Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruningcited by 20
  38. 2021Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Frameworkcited by 19

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 7.

18 papers.

Sort:

By area:

  1. 2025MemOS: A Memory OS for AI Systemcited by 129
  2. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 68
  3. 2025Dissecting Bias in LLMs: A Mechanistic Interpretability Perspectivecited by 14
  4. 2025Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMscited by 7
  5. 2025Mitigating Biases in Language Models via Bias Unlearningcited by 6
  6. 2025No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biasescited by 6
  7. 2025LLM Bias Detection and Mitigation through the Lens of Desired Distributionscited by 4
  8. 2025BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responsescited by 2
  9. 2026Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Studycited by 2
  10. 2026Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Modelscited by 2
  11. 2025BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injectioncited by 1
  12. 2025Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Modelscited by 1
6 more
  1. 2026Tracing Stereotypes in Pre-trained Transformers: From Biased Neurons to Fairer Modelscited by 1
  2. 2025A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functionscited by 0
  3. 2026Knowing Bias, Doing Better: Mitigating Social Bias in LLMs via Know-Bias Neuron Enhancementcited by 0
  4. 2026CoCoA: Context-Conditional Cultural Alignment for Large Language Modelscited by 0
  5. 2026Debias It Yourself: Teaching LLMs Cognitive Bias Mitigation Interventionscited by 0
  6. 2026Acmite: Mitigating Gender Bias in LLMs through Concept-Guided Mutual Informationcited by 0