# FairFlow: Mitigating Dataset Biases through Undecided Learning

arXiv: https://arxiv.org/abs/2503.17632 · HTML: https://mlsanity.com/paper/2503.17632/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 40 of its 68 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Stress Test Evaluation for Natural Language Inference](https://mlsanity.com/paper/1806.00692/) (2018)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Towards Debiasing NLU Models from Unknown Biases](https://mlsanity.com/paper/2009.12303/) (2020)
- [Competency Problems: On Finding and Removing Artifacts in Language Data](https://mlsanity.com/paper/2104.08646/) (2021)
- [Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance](https://mlsanity.com/paper/2005.00315/) (2020)
- [Learning from others' mistakes: Avoiding dataset biases without modeling them](https://mlsanity.com/paper/2012.01300/) (2020)
- [FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders](https://mlsanity.com/paper/2103.06413/) (2021)
- [UnNatural Language Inference](https://mlsanity.com/paper/2101.00010/) (2020)
- [Natural Language Understanding with the Quora Question Pairs Dataset](https://mlsanity.com/paper/1907.01041/) (2019)
- [Learning Debiased Classifier with Biased Committee](https://mlsanity.com/paper/2206.10843/) (2022)
- [Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference](https://mlsanity.com/paper/1907.04380/) (2019)
- [Balancing out Bias: Achieving Fairness Through Balanced Training](https://mlsanity.com/paper/2109.08253/) (2021)
- [Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answering](https://mlsanity.com/paper/2310.06238/) (2023)
- [Group Robust Classification Without Any Group Information](https://mlsanity.com/paper/2310.18555/) (2023)
- [End-to-End Self-Debiasing Framework for Robust NLU Training](https://mlsanity.com/paper/2109.02071/) (2021)

## Abstract (arXiv)

Language models are prone to dataset biases, known as shortcuts and spurious correlations in data, which often result in performance drop on new data. We present a new debiasing framework called ``FairFlow'' that mitigates dataset biases by learning to be undecided in its predictions for data samples or representations associated with known or unknown biases. The framework introduces two key components: a suite of data and model perturbation operations that generate different biased views of input samples, and a contrastive objective that learns debiased and robust representations from the resulting biased views of samples. Experiments show that FairFlow outperforms existing debiasing methods, particularly against out-of-domain and hard test samples without compromising the in-domain performance