# Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features

arXiv: https://arxiv.org/abs/2308.08482 · HTML: https://mlsanity.com/paper/2308.08482/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 13 Aug 2023.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [On the Fairness of Disentangled Representations](https://mlsanity.com/paper/1905.13662/) (2019)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [Can contrastive learning avoid shortcut solutions?](https://mlsanity.com/paper/2106.11230/) (2021)
- [Interventional Few-Shot Learning](https://mlsanity.com/paper/2009.13000/) (2020)
- [Visual Commonsense R-CNN](https://mlsanity.com/paper/2002.12204/) (2020)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models](https://mlsanity.com/paper/2103.06922/) (2021)
- [Two Causal Principles for Improving Visual Dialog](https://mlsanity.com/paper/1911.10496/) (2019)
- [Fair Contrastive Learning for Facial Attribute Classification](https://mlsanity.com/paper/2203.16209/) (2022)
- [Fairness via Representation Neutralization](https://mlsanity.com/paper/2106.12674/) (2021)
- [Fair Feature Distillation for Visual Recognition](https://mlsanity.com/paper/2106.04411/) (2021)
- [Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification](https://mlsanity.com/paper/1907.09013/) (2019)

## Built on by (most cited first, 2 of 2)

- [Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse](https://mlsanity.com/paper/2405.05587/) (2024)
- [Marginal Debiased Network for Fair Visual Recognition](https://mlsanity.com/paper/2401.02150/) (2024)

## Abstract (arXiv)

Machine learning models often learn to make predictions that rely on sensitive social attributes like gender and race, which poses significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice. Existing work tackles this issue by minimizing the employed information about social attributes in models for debiasing. However, the high correlation between target task and these social attributes makes learning on the target task incompatible with debiasing. Given that model bias arises due to the learning of bias features (i.e., gender) that help target task optimization, we explore the following research question: Can we leverage shortcut features to replace the role of bias feature in target task optimization for debiasing? To this end, we propose Shortcut Debiasing, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference. The key idea of Shortcut Debiasing is to design controllable shortcut features to on one hand replace bias features in contributing to the target task during the training stage, and on the other hand be easily removed by intervention during the inference stage. This guarantees the learning of the target task does not hinder the elimination of bias features. We apply Shortcut Debiasing to several benchmark datasets, and achieve significant improvements over the state-of-the-art debiasing methods in both accuracy and fairness.