# Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach

arXiv: https://arxiv.org/abs/1904.05514 · HTML: https://mlsanity.com/paper/1904.05514/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2019.

## Built on by (most cited first, 30 of 48)

- [Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation](https://mlsanity.com/paper/2101.04108/) (2021)
- [Fair Normalizing Flows](https://mlsanity.com/paper/2106.05937/) (2021)
- [Learning Fair Representation via Distributional Contrastive Disentanglement](https://mlsanity.com/paper/2206.08743/) (2022)
- [Invariant Causal Mechanisms through Distribution Matching](https://mlsanity.com/paper/2206.11646/) (2022)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [FARE: Provably Fair Representation Learning with Practical Certificates](https://mlsanity.com/paper/2210.07213/) (2022)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)
- [Utility-Fairness Trade-Offs and How to Find Them](https://mlsanity.com/paper/2404.09454/) (2024)
- [Privacy-Preserving Action Recognition via Motion Difference Quantization](https://mlsanity.com/paper/2208.02459/) (2022)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Demographic Fairness in Biometric Systems: What do the Experts say?](https://mlsanity.com/paper/2105.14844/) (2021)
- [On Characterizing the Trade-off in Invariant Representation Learning](https://mlsanity.com/paper/2109.03386/) (2021)
- [CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglement](https://mlsanity.com/paper/2503.07938/) (2025)
- [Attributing Fair Decisions with Attention Interventions](https://mlsanity.com/paper/2109.03952/) (2021)
- [Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium](https://mlsanity.com/paper/2410.16432/) (2024)
- [AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning](https://mlsanity.com/paper/2112.01637/) (2021)
- [Scalable Infomin Learning](https://mlsanity.com/paper/2302.10701/) (2023)
- [AugFL: Augmenting Federated Learning with Pretrained Models](https://mlsanity.com/paper/2503.02154/) (2025)
- [Decouple-and-Sample: Protecting sensitive information in task agnostic data release](https://mlsanity.com/paper/2203.13204/) (2022)
- [Do learned representations respect causal relationships?](https://mlsanity.com/paper/2204.00762/) (2022)
- [Learning Robust and Privacy-Preserving Representations via Information Theory](https://mlsanity.com/paper/2412.11066/) (2024)
- [Adversarial Representation Learning With Closed-Form Solvers](https://mlsanity.com/paper/2109.05535/) (2021)
- [Distraction is All You Need for Fairness](https://mlsanity.com/paper/2203.07593/) (2022)
- [Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning](https://mlsanity.com/paper/2406.05686/) (2024)
- [Debiasing pipeline improves deep learning model generalization for X-ray based lung nodule detection](https://mlsanity.com/paper/2201.09563/) (2022)
- [SoFaiR: Single Shot Fair Representation Learning](https://mlsanity.com/paper/2204.12556/) (2022)
- [Classification Utility, Fairness, and Compactness via Tunable Information Bottleneck and R'enyi Measures](https://mlsanity.com/paper/2206.10043/) (2022)
- [Privacy Safe Representation Learning via Frequency Filtering Encoder](https://mlsanity.com/paper/2208.02482/) (2022)
- [Learning Fair Representations with High-Confidence Guarantees](https://mlsanity.com/paper/2310.15358/) (2023)
- [A Multi-Objective Evaluation Framework for Analyzing Utility-Fairness Trade-Offs in Machine Learning Systems](https://mlsanity.com/paper/2503.11120/) (2025)

## Abstract (arXiv)

Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from an image representation, which might compromise the privacy of the data owner. This paper investigates the problem of learning an image representation that minimizes such leakage of user information. We formulate the problem as an adversarial non-zero sum game of finding a good embedding function with two competing goals: to retain as much task dependent discriminative image information as possible, while simultaneously minimizing the amount of information, as measured by entropy, about other sensitive attributes of the user. We analyze the stability and convergence dynamics of the proposed formulation using tools from non-linear systems theory and compare to that of the corresponding adversarial zero-sum game formulation that optimizes likelihood as a measure of information content. Numerical experiments on UCI, Extended Yale B, CIFAR-10 and CIFAR-100 datasets indicate that our proposed approach is able to learn image representations that exhibit high task performance while mitigating leakage of predefined sensitive information.