# Learning Unbiased Representations via Mutual Information Backpropagation

arXiv: https://arxiv.org/abs/2003.06430 · HTML: https://mlsanity.com/paper/2003.06430/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2020.

## Built on by (most cited first, 25 of 25)

- [Beyond Normal: On the Evaluation of Mutual Information Estimators](https://mlsanity.com/paper/2306.11078/) (2023)
- [Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization](https://mlsanity.com/paper/2108.05449/) (2021)
- [Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning](https://mlsanity.com/paper/2106.02866/) (2021)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Information-Theoretic Bias Reduction via Causal View of Spurious Correlation](https://mlsanity.com/paper/2201.03121/) (2022)
- [Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility](https://mlsanity.com/paper/2207.04895/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [MMD-ReID: A Simple but Effective Solution for Visible-Thermal Person ReID](https://mlsanity.com/paper/2111.05059/) (2021)
- [Combating Unknown Bias with Effective Bias-Conflicting Scoring and Gradient Alignment](https://mlsanity.com/paper/2111.13108/) (2021)
- [Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition](https://mlsanity.com/paper/2111.04673/) (2021)
- [Information-Theoretic Bounds on The Removal of Attribute-Specific Bias From Neural Networks](https://mlsanity.com/paper/2310.04955/) (2023)
- [Structured Probabilistic Coding](https://mlsanity.com/paper/2312.13933/) (2023)
- [Group-robust Machine Unlearning](https://mlsanity.com/paper/2503.09330/) (2025)
- [Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play](https://mlsanity.com/paper/2302.05807/) (2023)
- [Representation Learning with Conditional Information Flow Maximization](https://mlsanity.com/paper/2406.05510/) (2024)
- [Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation](https://mlsanity.com/paper/2507.20284/) (2025)
- [Measuring Statistical Dependencies via Maximum Norm and Characteristic Functions](https://mlsanity.com/paper/2208.07934/) (2022)
- [Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagery](https://mlsanity.com/paper/2211.08672/) (2022)
- [EnfoMax: Domain Entropy and Mutual Information Maximization for Domain Generalized Face Anti-spoofing](https://mlsanity.com/paper/2302.08674/) (2023)
- [SABAF: Removing Strong Attribute Bias from Neural Networks with Adversarial Filtering](https://mlsanity.com/paper/2311.07141/) (2023)
- [Model Debiasing by Learnable Data Augmentation](https://mlsanity.com/paper/2408.04955/) (2024)
- [Delving into Identify-Emphasize Paradigm for Combating Unknown Bias](https://mlsanity.com/paper/2302.11414/) (2023)
- [Information Subtraction: Learning Representations for Conditional Entropy](https://mlsanity.com/paper/2501.02012/) (2025)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models](https://mlsanity.com/paper/2603.05582/) (2026)

## Abstract (arXiv)

We are interested in learning data-driven representations that can generalize well, even when trained on inherently biased data. In particular, we face the case where some attributes (bias) of the data, if learned by the model, can severely compromise its generalization properties. We tackle this problem through the lens of information theory, leveraging recent findings for a differentiable estimation of mutual information. We propose a novel end-to-end optimization strategy, which simultaneously estimates and minimizes the mutual information between the learned representation and the data attributes. When applied on standard benchmarks, our model shows comparable or superior classification performance with respect to state-of-the-art approaches. Moreover, our method is general enough to be applicable to the problem of ``algorithmic fairness'', with competitive results.