# Invariant Representations through Adversarial Forgetting

arXiv: https://arxiv.org/abs/1911.04060 · HTML: https://mlsanity.com/paper/1911.04060/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 20 of 20)

- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation](https://mlsanity.com/paper/2101.04108/) (2021)
- [Fair Normalizing Flows](https://mlsanity.com/paper/2106.05937/) (2021)
- [Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness](https://mlsanity.com/paper/2211.11109/) (2022)
- [Robust Concept Erasure via Kernelized Rate-Distortion Maximization](https://mlsanity.com/paper/2312.00194/) (2023)
- [Attributing Fair Decisions with Attention Interventions](https://mlsanity.com/paper/2109.03952/) (2021)
- [Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium](https://mlsanity.com/paper/2410.16432/) (2024)
- [Scalable Infomin Learning](https://mlsanity.com/paper/2302.10701/) (2023)
- [Fair Representations by Compression](https://mlsanity.com/paper/2105.14044/) (2021)
- [CAT: Controllable Attribute Translation for Fair Facial Attribute Classification](https://mlsanity.com/paper/2209.06850/) (2022)
- [Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition](https://mlsanity.com/paper/2111.04673/) (2021)
- [Training Domain-invariant Object Detector Faster with Feature Replay and Slow Learner](https://mlsanity.com/paper/2105.14693/) (2021)
- [Distraction is All You Need for Fairness](https://mlsanity.com/paper/2203.07593/) (2022)
- [SoFaiR: Single Shot Fair Representation Learning](https://mlsanity.com/paper/2204.12556/) (2022)
- [Learning Fair Representations with High-Confidence Guarantees](https://mlsanity.com/paper/2310.15358/) (2023)
- [Nearest-Neighbor Density Estimation for Dependency Suppression](https://mlsanity.com/paper/2603.04224/) (2026)
- [Simple and Effective Specialized Representations for Fair Classifiers](https://mlsanity.com/paper/2505.11740/) (2025)
- [FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data](https://mlsanity.com/paper/2510.02017/) (2025)
- [Fair Representation Learning with Controllable High Confidence Guarantees via Adversarial Inference](https://mlsanity.com/paper/2510.21017/) (2025)

## Abstract (arXiv)

We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism. We show that the forgetting mechanism serves as an information-bottleneck, which is manipulated by the adversarial training to learn invariance to unwanted factors. Empirical results show that the proposed framework achieves state-of-the-art performance at learning invariance in both nuisance and bias settings on a diverse collection of datasets and tasks.