Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation
Not on the map. Why?
Abstract (arXiv)
Mitigation of biases, such as language models' reliance on gender stereotypes, is a crucial endeavor required for the creation of reliable and useful language technology. The crucial aspect of debiasing is to ensure that the models preserve their versatile capabilities, including their ability to solve language tasks and equitably represent various genders. To address this issue, we introduce a streamlined Dual Dabiasing Algorithm through Model Adaptation (2DAMA). Novel Dual Debiasing enables robust reduction of stereotypical bias while preserving desired factual gender information encoded by language models. We show that 2DAMA effectively reduces gender bias in English and is one of the first approaches facilitating the mitigation of stereotypical tendencies in translation. The proposed method's key advantage is the preservation of factual gender cues, which are useful in a wide range of natural language processing tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 17 Jan 2025.
Builds on
29 of 41 references.
References · 29
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2022Mass-Editing Memory in a Transformercited by 976
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
- 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
- 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
- 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
17 more
- 2023LEACE: Perfect linear concept erasure in closed formcited by 254
- 2019Evaluating Gender Bias in Machine Translationcited by 247
- 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
- 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
- 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
- 2021Gender Bias in Machine Translationcited by 116
- 2022Linear Adversarial Concept Erasurecited by 98
- 2021A Survey on Gender Bias in Natural Language Processingcited by 89
- 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
- 2021Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translationcited by 62
- 2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
- 2020Gender Coreference and Bias Evaluation at WMT 2020cited by 21
- 2022Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Informationcited by 16
- 2022The Birth of Bias: A case study on the evolution of gender bias in an English language modelcited by 16
- 2022Learning Fair Representations via Rate-Distortion Maximizationcited by 15
- 2022Analyzing Gender Representation in Multilingual Modelscited by 10
- 2023Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translationcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.