# Improving Evaluation of Debiasing in Image Classification

arXiv: https://arxiv.org/abs/2206.03680 · HTML: https://mlsanity.com/paper/2206.03680/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Analyzing the Behavior of Visual Question Answering Models](https://mlsanity.com/paper/1606.07356/) (2016)
- [Towards Debiasing NLU Models from Unknown Biases](https://mlsanity.com/paper/2009.12303/) (2020)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [Learning from others' mistakes: Avoiding dataset biases without modeling them](https://mlsanity.com/paper/2012.01300/) (2020)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [FR-Train: A Mutual Information-Based Approach to Fair and Robust Training](https://mlsanity.com/paper/2002.10234/) (2020)
- [Hard-Aware Deeply Cascaded Embedding](https://mlsanity.com/paper/1611.05720/) (2016)
- [Revisiting the Importance of Amplifying Bias for Debiasing](https://mlsanity.com/paper/2205.14594/) (2022)

## Built on by (most cited first, 1 of 1)

- [Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions](https://mlsanity.com/paper/2605.28780/) (2026)

## Abstract (arXiv)

Image classifiers often rely overly on peripheral attributes that have a strong correlation with the target class (i.e., dataset bias) when making predictions. Due to the dataset bias, the model correctly classifies data samples including bias attributes (i.e., bias-aligned samples) while failing to correctly predict those without bias attributes (i.e., bias-conflicting samples). Recently, a myriad of studies focus on mitigating such dataset bias, the task of which is referred to as debiasing. However, our comprehensive study indicates several issues need to be improved when conducting evaluation of debiasing in image classification. First, most of the previous studies do not specify how they select their hyper-parameters and model checkpoints (i.e., tuning criterion). Second, the debiasing studies until now evaluated their proposed methods on datasets with excessively high bias-severities, showing degraded performance on datasets with low bias severity. Third, the debiasing studies do not share consistent experimental settings (e.g., datasets and neural networks) which need to be standardized for fair comparisons. Based on such issues, this paper 1) proposes an evaluation metric `Align-Conflict (AC) score' for the tuning criterion, 2) includes experimental settings with low bias severity and shows that they are yet to be explored, and 3) unifies the standardized experimental settings to promote fair comparisons between debiasing methods. We believe that our findings and lessons inspire future researchers in debiasing to further push state-of-the-art performances with fair comparisons.