# Revisiting the Importance of Amplifying Bias for Debiasing

arXiv: https://arxiv.org/abs/2205.14594 · HTML: https://mlsanity.com/paper/2205.14594/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Analyzing the Behavior of Visual Question Answering Models](https://mlsanity.com/paper/1606.07356/) (2016)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [Lifespan Age Transformation Synthesis](https://mlsanity.com/paper/2003.09764/) (2020)
- [Neural Ensemble Search for Uncertainty Estimation and Dataset Shift](https://mlsanity.com/paper/2006.08573/) (2020)
- [When does Diversity Help Generalization in Classification Ensembles?](https://mlsanity.com/paper/1910.13631/) (2019)
- [Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural Networks](https://mlsanity.com/paper/2011.11486/) (2020)

## Built on by (most cited first, 19 of 19)

- [Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors](https://mlsanity.com/paper/2403.07366/) (2024)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations](https://mlsanity.com/paper/2304.14252/) (2023)
- [Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse](https://mlsanity.com/paper/2405.05587/) (2024)
- [Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness](https://mlsanity.com/paper/2310.06161/) (2023)
- [DeNetDM: Debiasing by Network Depth Modulation](https://mlsanity.com/paper/2403.19863/) (2024)
- [Enhancing Intrinsic Features for Debiasing via Investigating Class-Discerning Common Attributes in Bias-Contrastive Pair](https://mlsanity.com/paper/2404.19250/) (2024)
- [Improving Scene Text Recognition for Character-Level Long-Tailed Distribution](https://mlsanity.com/paper/2304.08592/) (2023)
- [Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing](https://mlsanity.com/paper/2502.09564/) (2025)
- [Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play](https://mlsanity.com/paper/2302.05807/) (2023)
- [Improving Bias Mitigation through Bias Experts in Natural Language Understanding](https://mlsanity.com/paper/2312.03577/) (2023)
- [Improving Evaluation of Debiasing in Image Classification](https://mlsanity.com/paper/2206.03680/) (2022)
- [Common-Sense Bias Modeling for Classification Tasks](https://mlsanity.com/paper/2401.13213/) (2024)
- [Model Debiasing by Learnable Data Augmentation](https://mlsanity.com/paper/2408.04955/) (2024)
- [CosFairNet:A Parameter-Space based Approach for Bias Free Learning](https://mlsanity.com/paper/2410.15094/) (2024)
- [Fine-Grained Class-Conditional Distribution Balancing for Debiased Learning](https://mlsanity.com/paper/2505.06831/) (2025)
- [BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers](https://mlsanity.com/paper/2605.28450/) (2026)
- [Towards Real-world Debiasing: Rethinking Evaluation, Challenge, and Solution](https://mlsanity.com/paper/2405.15240/) (2024)
- [Shortcut Features as Top Eigenfunctions of NTK: A Linear Neural Network Case and More](https://mlsanity.com/paper/2602.03066/) (2026)

## Abstract (arXiv)

In image classification, "debiasing" aims to train a classifier to be less susceptible to dataset bias, the strong correlation between peripheral attributes of data samples and a target class. For example, even if the frog class in the dataset mainly consists of frog images with a swamp background (i.e., bias-aligned samples), a debiased classifier should be able to correctly classify a frog at a beach (i.e., bias-conflicting samples). Recent debiasing approaches commonly use two components for debiasing, a biased model fB and a debiased model fD. fB is trained to focus on bias-aligned samples (i.e., overfitted to the bias) while fD is mainly trained with bias-conflicting samples by concentrating on samples which fB fails to learn, leading fD to be less susceptible to the dataset bias. While the state-of-the-art debiasing techniques have aimed to better train fD, we focus on training fB, an overlooked component until now. Our empirical analysis reveals that removing the bias-conflicting samples from the training set for fB is important for improving the debiasing performance of fD. This is due to the fact that the bias-conflicting samples work as noisy samples for amplifying the bias for fB since those samples do not include the bias attribute. To this end, we propose a simple yet effective data sample selection method which removes the bias-conflicting samples to construct a bias-amplified dataset for training fB. Our data sample selection method can be directly applied to existing reweighting-based debiasing approaches, obtaining consistent performance boost and achieving the state-of-the-art performance on both synthetic and real-world datasets.