# Learning unbiased features

arXiv: https://arxiv.org/abs/1412.5244 · HTML: https://mlsanity.com/paper/1412.5244/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2014.

## Built on by (most cited first, 10 of 10)

- [Removing biased data to improve fairness and accuracy](https://mlsanity.com/paper/2102.03054/) (2021)
- [Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning](https://mlsanity.com/paper/2203.12748/) (2022)
- [Fair Representations by Compression](https://mlsanity.com/paper/2105.14044/) (2021)
- [Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasets](https://mlsanity.com/paper/2203.15234/) (2022)
- [Learning Invariant Representations using Inverse Contrastive Loss](https://mlsanity.com/paper/2102.08343/) (2021)
- [Pooling Image Datasets With Multiple Covariate Shift and Imbalance](https://mlsanity.com/paper/2403.02598/) (2024)
- [Training Domain-invariant Object Detector Faster with Feature Replay and Slow Learner](https://mlsanity.com/paper/2105.14693/) (2021)
- [Harnessing value from data science in business: ensuring explainability and fairness of solutions](https://mlsanity.com/paper/2108.07714/) (2021)
- [Weakly Supervised Invariant Representation Learning Via Disentangling Known and Unknown Nuisance Factors](https://mlsanity.com/paper/2209.06827/) (2022)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)

## Abstract (arXiv)

A key element in transfer learning is representation learning; if representations can be developed that expose the relevant factors underlying the data, then new tasks and domains can be learned readily based on mappings of these salient factors. We propose that an important aim for these representations are to be unbiased. Different forms of representation learning can be derived from alternative definitions of unwanted bias, e.g., bias to particular tasks, domains, or irrelevant underlying data dimensions. One very useful approach to estimating the amount of bias in a representation comes from maximum mean discrepancy (MMD) [5], a measure of distance between probability distributions. We are not the first to suggest that MMD can be a useful criterion in developing representations that apply across multiple domains or tasks [1]. However, in this paper we describe a number of novel applications of this criterion that we have devised, all based on the idea of developing unbiased representations. These formulations include: a standard domain adaptation framework; a method of learning invariant representations; an approach based on noise-insensitive autoencoders; and a novel form of generative model.