# On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations

arXiv: https://arxiv.org/abs/2204.12708 · HTML: https://mlsanity.com/paper/2204.12708/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 107 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 30 of 63)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)

## Built on by (most cited first, 19 of 19)

- [Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens](https://mlsanity.com/paper/2210.14011/) (2022)
- [TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models](https://mlsanity.com/paper/2304.08821/) (2023)
- [Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases](https://mlsanity.com/paper/2305.18917/) (2023)
- [Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection](https://mlsanity.com/paper/2406.16846/) (2024)
- [Principles from Clinical Research for NLP Model Generalization](https://mlsanity.com/paper/2311.03663/) (2023)
- [Compare without Despair: Reliable Preference Evaluation with Generation Separability](https://mlsanity.com/paper/2407.01878/) (2024)
- [Beyond Benchmarks: On The False Promise of AI Regulation](https://mlsanity.com/paper/2501.15693/) (2025)
- [The Impact of Coreset Selection on Spurious Correlations and Group Robustness](https://mlsanity.com/paper/2507.11690/) (2025)
- [Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference](https://mlsanity.com/paper/2211.03862/) (2022)
- [Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution](https://mlsanity.com/paper/2305.15389/) (2023)
- [Stubborn Lexical Bias in Data and Models](https://mlsanity.com/paper/2306.02190/) (2023)
- [Optimizing importance weighting in the presence of sub-population shifts](https://mlsanity.com/paper/2410.14315/) (2024)
- [HBO: Hierarchical Balancing Optimization for Fine-Tuning Large Language Models](https://mlsanity.com/paper/2505.12300/) (2025)
- [On Degrees of Freedom in Defining and Testing Natural Language Understanding](https://mlsanity.com/paper/2305.15130/) (2023)
- [Cross-modal Associations in Vision and Language Models: Revisiting the Bouba-Kiki Effect](https://mlsanity.com/paper/2507.10013/) (2025)
- [Characterizing Stereotypical Bias from Privacy-preserving Pre-Training](https://mlsanity.com/paper/2407.00764/) (2024)
- [Effective Guidance for Model Attention with Simple Yes-no Annotations](https://mlsanity.com/paper/2410.22312/) (2024)
- [On the Shortcut Learning in Multilingual Neural Machine Translation](https://mlsanity.com/paper/2411.10581/) (2024)
- [Representation Invariance and Allocation: When Subgroup Balance Matters](https://mlsanity.com/paper/2512.09496/) (2025)

## Abstract (arXiv)

Recent work has shown that deep learning models in NLP are highly sensitive to low-level correlations between simple features and specific output labels, leading to overfitting and lack of generalization. To mitigate this problem, a common practice is to balance datasets by adding new instances or by filtering out "easy" instances (Sakaguchi et al., 2020), culminating in a recent proposal to eliminate single-word correlations altogether (Gardner et al., 2021). In this opinion paper, we identify that despite these efforts, increasingly-powerful models keep exploiting ever-smaller spurious correlations, and as a result even balancing all single-word features is insufficient for mitigating all of these correlations. In parallel, a truly balanced dataset may be bound to "throw the baby out with the bathwater" and miss important signal encoding common sense and world knowledge. We highlight several alternatives to dataset balancing, focusing on enhancing datasets with richer contexts, allowing models to abstain and interact with users, and turning from large-scale fine-tuning to zero- or few-shot setups.