# Information-Theoretic Bias Reduction via Causal View of Spurious Correlation

arXiv: https://arxiv.org/abs/2201.03121 · HTML: https://mlsanity.com/paper/2201.03121/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019)
- [Learning Meta Face Recognition in Unseen Domains](https://mlsanity.com/paper/2003.07733/) (2020)
- [Unsupervised Learning of Debiased Representations with Pseudo-Attributes](https://mlsanity.com/paper/2108.02943/) (2021)
- [DisturbLabel: Regularizing CNN on the Loss Layer](https://mlsanity.com/paper/1605.00055/) (2016)
- [Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual Debiasing](https://mlsanity.com/paper/2007.13632/) (2020)
- [Learning Unbiased Representations via Mutual Information Backpropagation](https://mlsanity.com/paper/2003.06430/) (2020)
- [Learning Debiased and Disentangled Representations for Semantic Segmentation](https://mlsanity.com/paper/2111.00531/) (2021)
- [Data-Efficient Mutual Information Neural Estimator](https://mlsanity.com/paper/1905.03319/) (2019)

## Built on by (most cited first, 15 of 15)

- [Meta Spatio-Temporal Debiasing for Video Scene Graph Generation](https://mlsanity.com/paper/2207.11441/) (2022)
- [Shift-Robust Molecular Relational Learning with Causal Substructure](https://mlsanity.com/paper/2305.18451/) (2023)
- [How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features](https://mlsanity.com/paper/2305.12100/) (2023)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Breaking Correlation Shift via Conditional Invariant Regularizer](https://mlsanity.com/paper/2207.06687/) (2022)
- [Towards Adversarial Robustness via Debiased High-Confidence Logit Alignment](https://mlsanity.com/paper/2408.06079/) (2024)
- [Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation](https://mlsanity.com/paper/2507.20284/) (2025)
- [Will the Inclusion of Generated Data Amplify Bias Across Generations in Future Image Classification Models?](https://mlsanity.com/paper/2410.10160/) (2024)
- [The Role of Model Confidence on Bias Effects in Measured Uncertainties for Vision-Language Models](https://mlsanity.com/paper/2506.16724/) (2025)
- [Spuriosity Rankings for Free: A Simple Framework for Last Layer Retraining Based on Object Detection](https://mlsanity.com/paper/2311.00079/) (2023)
- [Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation](https://mlsanity.com/paper/2410.05345/) (2024)
- [Effective Guidance for Model Attention with Simple Yes-no Annotations](https://mlsanity.com/paper/2410.22312/) (2024)
- [Re-evaluating Group Robustness via Adaptive Class-Specific Scaling](https://mlsanity.com/paper/2412.15311/) (2024)
- [Mitigating Shortcut Learning via Feature Disentanglement in Medical Imaging: A Benchmark Study](https://mlsanity.com/paper/2602.18502/) (2026)
- [Adaptive Causal Alignment for High-Confidence Adversarial Training](https://mlsanity.com/paper/2606.03925/) (2026)

## Abstract (arXiv)

We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information. Although several bias measurement methods have been proposed and widely investigated to achieve algorithmic fairness in various tasks such as face recognition, their accuracy- or logit-based metrics are susceptible to leading to trivial prediction score adjustment rather than fundamental bias reduction. Hence, we design a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss derived by the proposed information-theoretic bias measurement approach. In addition, we present a simple yet effective unsupervised debiasing technique based on stochastic label noise, which does not require the explicit supervision of bias information. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios through extensive experiments on multiple standard benchmarks.