mlsanityData to 14 Oct 2026

Information-Theoretic Bias Reduction via Causal View of Spurious Correlation

arXiv 2201.03121 ↗First posted 10 Jan 2022cs.LG cs.AI cs.CYcited by 15

Not on the map. Why?

Abstract (arXiv)

We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information. Although several bias measurement methods have been proposed and widely investigated to achieve algorithmic fairness in various tasks such as face recognition, their accuracy- or logit-based metrics are susceptible to leading to trivial prediction score adjustment rather than fundamental bias reduction. Hence, we design a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss derived by the proposed information-theoretic bias measurement approach. In addition, we present a simple yet effective unsupervised debiasing technique based on stochastic label noise, which does not require the explicit supervision of bias information. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios through extensive experiments on multiple standard benchmarks.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 28 of its 51 references that are in the map. First posted on arXiv 10 Jan 2022.

Builds on

28 of 51 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  6. 2019Invariant Risk Minimizationcited by 1,854
  7. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  9. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  10. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  11. 2018MINE: Mutual Information Neural Estimationcited by 683
  12. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
16 more
  1. 2019On Variational Bounds of Mutual Informationcited by 559
  2. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  3. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  4. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  5. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  6. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 209
  7. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 185
  8. 2019REPAIR: Removing Representation Bias by Dataset Resamplingcited by 157
  9. 2019FairFace: Face Attribute Dataset for Balanced Race, Gender, and Agecited by 87
  10. 2020Learning Meta Face Recognition in Unseen Domainscited by 61
  11. 2021Unsupervised Learning of Debiased Representations with Pseudo-Attributescited by 44
  12. 2016DisturbLabel: Regularizing CNN on the Loss Layercited by 41
  13. 2020Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual Debiasingcited by 32
  14. 2020Learning Unbiased Representations via Mutual Information Backpropagationcited by 25
  15. 2021Learning Debiased and Disentangled Representations for Semantic Segmentationcited by 16
  16. 2019Data-Efficient Mutual Information Neural Estimatorcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 5; 2025: 3; 2026: 2.

15 papers.

Sort:

By area:

  1. 2022Meta Spatio-Temporal Debiasing for Video Scene Graph Generationcited by 20
  2. 2023Shift-Robust Molecular Relational Learning with Causal Substructurecited by 15
  3. 2023How Spurious Features Are Memorized: Precise Analysis for Random and NTK Featurescited by 11
  4. 2025Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterizationcited by 9
  5. 2022Breaking Correlation Shift via Conditional Invariant Regularizercited by 7
  6. 2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
  7. 2025Controllable Feature Whitening for Hyperparameter-Free Bias Mitigationcited by 2
  8. 2024Will the Inclusion of Generated Data Amplify Bias Across Generations in Future Image Classification Models?cited by 1
  9. 2025The Role of Model Confidence on Bias Effects in Measured Uncertainties for Vision-Language Modelscited by 1
  10. 2023Spuriosity Rankings for Free: A Simple Framework for Last Layer Retraining Based on Object Detectioncited by 0
  11. 2024Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotationcited by 0
  12. 2024Effective Guidance for Model Attention with Simple Yes-no Annotationscited by 0
3 more
  1. 2024Re-evaluating Group Robustness via Adaptive Class-Specific Scalingcited by 0
  2. 2026Mitigating Shortcut Learning via Feature Disentanglement in Medical Imaging: A Benchmark Studycited by 0
  3. 2026Adaptive Causal Alignment for High-Confidence Adversarial Trainingcited by 0