# Right for the Right Latent Factors: Debiasing Generative Models via Disentanglement

arXiv: https://arxiv.org/abs/2202.00391 · HTML: https://mlsanity.com/paper/2202.00391/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [Interpretations are useful: penalizing explanations to align neural networks with prior knowledge](https://mlsanity.com/paper/1909.13584/) (2019)
- [On the Fairness of Disentangled Representations](https://mlsanity.com/paper/1905.13662/) (2019)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations](https://mlsanity.com/paper/1705.08841/) (2017)
- [Are Disentangled Representations Helpful for Abstract Visual Reasoning?](https://mlsanity.com/paper/1905.12506/) (2019)
- [Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)](https://mlsanity.com/paper/2001.04872/) (2020)
- [A Deeper Look at Dataset Bias](https://mlsanity.com/paper/1505.01257/) (2015)
- [Weakly Supervised Disentanglement with Guarantees](https://mlsanity.com/paper/1910.09772/) (2019)
- [Disentangled Representation Learning in Cardiac Image Analysis](https://mlsanity.com/paper/1903.09467/) (2019)
- [Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs](https://mlsanity.com/paper/1801.05453/) (2018)
- [Disentangling Factors of Variation Using Few Labels](https://mlsanity.com/paper/1905.01258/) (2019)

## Built on by (most cited first, 2 of 2)

- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)

## Abstract (arXiv)

A key assumption of most statistical machine learning methods is that they have access to independent samples from the distribution of data they encounter at test time. As such, these methods often perform poorly in the face of biased data, which breaks this assumption. In particular, machine learning models have been shown to exhibit Clever-Hans-like behaviour, meaning that spurious correlations in the training set are inadvertently learnt. A number of works have been proposed to revise deep classifiers to learn the right correlations. However, generative models have been overlooked so far. We observe that generative models are also prone to Clever-Hans-like behaviour. To counteract this issue, we propose to debias generative models by disentangling their internal representations, which is achieved via human feedback. Our experiments show that this is effective at removing bias even when human feedback covers only a small fraction of the desired distribution. In addition, we achieve strong disentanglement results in a quantitative comparison with recent methods.