# End-to-End Bias Mitigation by Modelling Biases in Corpora

arXiv: https://arxiv.org/abs/1909.06321 · HTML: https://mlsanity.com/paper/1909.06321/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2019.

## Built on by (most cited first, 30 of 128)

- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [Impact of Pretraining Term Frequencies on Few-Shot Reasoning](https://mlsanity.com/paper/2202.07206/) (2022)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations](https://mlsanity.com/paper/2306.04618/) (2023)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Competency Problems: On Finding and Removing Artifacts in Language Data](https://mlsanity.com/paper/2104.08646/) (2021)
- [Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models](https://mlsanity.com/paper/2103.06922/) (2021)
- [Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.05199/) (2023)
- [Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets](https://mlsanity.com/paper/2203.12942/) (2022)
- [Variational Information Bottleneck for Effective Low-Resource Fine-Tuning](https://mlsanity.com/paper/2106.05469/) (2021)
- [Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions](https://mlsanity.com/paper/2205.00415/) (2022)
- [Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters](https://mlsanity.com/paper/2204.07447/) (2022)
- [Greedy Gradient Ensemble for Robust Visual Question Answering](https://mlsanity.com/paper/2107.12651/) (2021)
- [DISCO: Distilling Counterfactuals with Large Language Models](https://mlsanity.com/paper/2212.10534/) (2022)
- [Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment](https://mlsanity.com/paper/2403.02738/) (2024)
- [Supervising Model Attention with Human Explanations for Robust Natural Language Inference](https://mlsanity.com/paper/2104.08142/) (2021)
- [Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens](https://mlsanity.com/paper/2210.14011/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [From Shortcuts to Triggers: Backdoor Defense with Denoised PoE](https://mlsanity.com/paper/2305.14910/) (2023)
- [Explanation-based Finetuning Makes Models More Robust to Spurious Cues](https://mlsanity.com/paper/2305.04990/) (2023)
- [ProtT3: Protein-to-Text Generation for Text-based Protein Understanding](https://mlsanity.com/paper/2405.12564/) (2024)
- ["Garbage In, Garbage Out" Revisited: What Do Machine Learning Application Papers Report About Human-Labeled Training Data?](https://mlsanity.com/paper/2107.02278/) (2021)
- [End-to-End Self-Debiasing Framework for Robust NLU Training](https://mlsanity.com/paper/2109.02071/) (2021)

## Abstract (arXiv)

Several recent studies have shown that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models that fail to generalize to out-of-domain datasets and are likely to perform poorly in real-world scenarios. We propose two learning strategies to train neural models, which are more robust to such biases and transfer better to out-of-domain datasets. The biases are specified in terms of one or more bias-only models, which learn to leverage the dataset biases. During training, the bias-only models' predictions are used to adjust the loss of the base model to reduce its reliance on biases by down-weighting the biased examples and focusing the training on the hard examples. We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data. Results show that our debiasing methods greatly improve robustness in all settings and better transfer to other textual entailment datasets. Our code and data are publicly available in urlhttps://github.com/rabeehk/robust-nli.