# Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation

arXiv: https://arxiv.org/abs/2501.10150 · HTML: https://mlsanity.com/paper/2501.10150/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 17 Jan 2025.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem](https://mlsanity.com/paper/2004.04498/) (2020)
- [Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation](https://mlsanity.com/paper/2109.03858/) (2021)
- [AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks](https://mlsanity.com/paper/2205.00305/) (2022)
- [Gender Coreference and Bias Evaluation at WMT 2020](https://mlsanity.com/paper/2010.06018/) (2020)
- [Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information](https://mlsanity.com/paper/2206.10744/) (2022)
- [The Birth of Bias: A case study on the evolution of gender bias in an English language model](https://mlsanity.com/paper/2207.10245/) (2022)
- [Learning Fair Representations via Rate-Distortion Maximization](https://mlsanity.com/paper/2202.00035/) (2022)
- [Analyzing Gender Representation in Multilingual Models](https://mlsanity.com/paper/2204.09168/) (2022)
- [Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation](https://mlsanity.com/paper/2309.12491/) (2023)

## Built on by (most cited first, 2 of 2)

- [Preserving Task-Relevant Information Under Linear Concept Removal](https://mlsanity.com/paper/2506.10703/) (2025)
- [GKnow: Measuring the Entanglement of Gender Bias and Factual Gender](https://mlsanity.com/paper/2605.12299/) (2026)

## Abstract (arXiv)

Mitigation of biases, such as language models' reliance on gender stereotypes, is a crucial endeavor required for the creation of reliable and useful language technology. The crucial aspect of debiasing is to ensure that the models preserve their versatile capabilities, including their ability to solve language tasks and equitably represent various genders. To address this issue, we introduce a streamlined Dual Dabiasing Algorithm through Model Adaptation (2DAMA). Novel Dual Debiasing enables robust reduction of stereotypical bias while preserving desired factual gender information encoded by language models. We show that 2DAMA effectively reduces gender bias in English and is one of the first approaches facilitating the mitigation of stereotypical tendencies in translation. The proposed method's key advantage is the preservation of factual gender cues, which are useful in a wide range of natural language processing tasks.