# Discover the Unknown Biased Attribute of an Image Classifier

arXiv: https://arxiv.org/abs/2104.14556 · HTML: https://mlsanity.com/paper/2104.14556/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 54 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [On the Fairness of Disentangled Representations](https://mlsanity.com/paper/1905.13662/) (2019)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement](https://mlsanity.com/paper/2008.10599/) (2020)
- [Learning Disentangled Joint Continuous and Discrete Representations](https://mlsanity.com/paper/1804.00104/) (2018)
- [Fairness by Learning Orthogonal Disentangled Representations](https://mlsanity.com/paper/2003.05707/) (2020)
- [Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias](https://mlsanity.com/paper/1906.06439/) (2019)

## Built on by (most cited first, 30 of 46)

- [A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others](https://mlsanity.com/paper/2212.04825/) (2022)
- [Distilling Model Failures as Directions in Latent Space](https://mlsanity.com/paper/2206.14754/) (2022)
- [Discover and Cure: Concept-aware Mitigation of Spurious Correlation](https://mlsanity.com/paper/2305.00650/) (2023)
- [Discovering and Mitigating Visual Biases through Keyword Explanation](https://mlsanity.com/paper/2301.11104/) (2023)
- [Discover and Mitigate Unknown Biases with Debiasing Alternate Networks](https://mlsanity.com/paper/2207.10077/) (2022)
- [LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images](https://mlsanity.com/paper/2305.19164/) (2023)
- [STEEX: Steering Counterfactual Explanations with Semantics](https://mlsanity.com/paper/2111.09094/) (2021)
- [Invariant Feature Learning for Generalized Long-Tailed Classification](https://mlsanity.com/paper/2207.09504/) (2022)
- [StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2203.15799/) (2022)
- [On Learning Fairness and Accuracy on Multiple Subgroups](https://mlsanity.com/paper/2210.10837/) (2022)
- [Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data](https://mlsanity.com/paper/2310.16524/) (2023)
- [Evidence for Hypodescent in Visual Semantic AI](https://mlsanity.com/paper/2205.10764/) (2022)
- [OCTET: Object-aware Counterfactual Explanations](https://mlsanity.com/paper/2211.12380/) (2022)
- [Zero-shot Model Diagnosis](https://mlsanity.com/paper/2303.15441/) (2023)
- [Mitigating Dataset Bias by Using Per-sample Gradient](https://mlsanity.com/paper/2205.15704/) (2022)
- [On Modality Bias Recognition and Reduction](https://mlsanity.com/paper/2202.12690/) (2022)
- [Latent Space Smoothing for Individually Fair Representations](https://mlsanity.com/paper/2111.13650/) (2021)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Identification of Systematic Errors of Image Classifiers on Rare Subgroups](https://mlsanity.com/paper/2303.05072/) (2023)
- [Learning to Split for Automatic Bias Detection](https://mlsanity.com/paper/2204.13749/) (2022)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Discover and Mitigate Multiple Biased Subgroups in Image Classifiers](https://mlsanity.com/paper/2403.12777/) (2024)
- [medXGAN: Visual Explanations for Medical Classifiers through a Generative Latent Space](https://mlsanity.com/paper/2204.05376/) (2022)
- [Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection](https://mlsanity.com/paper/2207.02400/) (2022)
- [Self-supervised debiasing using low rank regularization](https://mlsanity.com/paper/2210.05248/) (2022)
- [Learning Stable Classifiers by Transferring Unstable Features](https://mlsanity.com/paper/2106.07847/) (2021)
- [Towards a More Rigorous Science of Blindspot Discovery in Image Classification Models](https://mlsanity.com/paper/2207.04104/) (2022)
- [Intersectional Unfairness Discovery](https://mlsanity.com/paper/2405.20790/) (2024)
- [Explaining Image Classifiers Using Contrastive Counterfactuals in Generative Latent Spaces](https://mlsanity.com/paper/2206.05257/) (2022)
- [Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation](https://mlsanity.com/paper/2307.12219/) (2023)

## Abstract (arXiv)

Recent works find that AI algorithms learn biases from data. Therefore, it is urgent and vital to identify biases in AI algorithms. However, the previous bias identification pipeline overly relies on human experts to conjecture potential biases (e.g., gender), which may neglect other underlying biases not realized by humans. To help human experts better find the AI algorithms' biases, we study a new problem in this work -- for a classifier that predicts a target attribute of the input image, discover its unknown biased attribute. To solve this challenging problem, we use a hyperplane in the generative model's latent space to represent an image attribute; thus, the original problem is transformed to optimizing the hyperplane's normal vector and offset. We propose a novel total-variation loss within this framework as the objective function and a new orthogonalization penalty as a constraint. The latter prevents trivial solutions in which the discovered biased attribute is identical with the target or one of the known-biased attributes. Extensive experiments on both disentanglement datasets and real-world datasets show that our method can discover biased attributes and achieve better disentanglement w.r.t. target attributes. Furthermore, the qualitative results show that our method can discover unnoticeable biased attributes for various object and scene classifiers, proving our method's generalizability for detecting biased attributes in diverse domains of images. The code is available at https://git.io/J3kMh.