# Discovery and Separation of Features for Invariant Representation Learning

arXiv: https://arxiv.org/abs/1912.00646 · HTML: https://mlsanity.com/paper/1912.00646/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 4 of 4)

- [Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning](https://mlsanity.com/paper/2203.12748/) (2022)
- [Fair Representations by Compression](https://mlsanity.com/paper/2105.14044/) (2021)
- [Identifying and Disentangling Spurious Features in Pretrained Image Representations](https://mlsanity.com/paper/2306.12673/) (2023)
- [Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck](https://mlsanity.com/paper/2303.14096/) (2023)

## Abstract (arXiv)

Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization. We propose a framework for training robust neural networks that induces invariance to nuisances through learning to discover and separate predictive and nuisance factors of data. We present an information theoretic formulation of our approach, from which we derive training objectives and its connections with previous methods. Empirical results on a wide array of datasets show that the proposed framework achieves state-of-the-art performance, without requiring nuisance annotations during training.