# Measuring Disentanglement: A Review of Metrics

arXiv: https://arxiv.org/abs/2012.09276 · HTML: https://mlsanity.com/paper/2012.09276/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 30 of 70)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Generalizing Nonlinear ICA Beyond Structural Sparsity](https://mlsanity.com/paper/2311.00866/) (2023)
- [Towards Robust Metrics for Concept Representation Evaluation](https://mlsanity.com/paper/2301.10367/) (2023)
- [Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations](https://mlsanity.com/paper/2112.02290/) (2021)
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](https://mlsanity.com/paper/2302.02913/) (2023)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)
- [Non-negative Contrastive Learning](https://mlsanity.com/paper/2403.12459/) (2024)
- [Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects](https://mlsanity.com/paper/2211.02247/) (2022)
- [3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces](https://mlsanity.com/paper/2111.12448/) (2021)
- [Learning Disentangled Representations for Natural Language Definitions](https://mlsanity.com/paper/2210.02898/) (2022)
- [Disentangling representations in Restricted Boltzmann Machines without adversaries](https://mlsanity.com/paper/2206.11600/) (2022)
- [Disentanglement Analysis with Partial Information Decomposition](https://mlsanity.com/paper/2108.13753/) (2021)
- [Review of Disentanglement Approaches for Medical Applications -- Towards Solving the Gordian Knot of Generative Models in Healthcare](https://mlsanity.com/paper/2203.11132/) (2022)
- [Disentangled Source-Free Personalization for Facial Expression Recognition with Neutral Target Data](https://mlsanity.com/paper/2503.20771/) (2025)
- [Disentangled Generative Models for Robust Prediction of System Dynamics](https://mlsanity.com/paper/2108.11684/) (2021)
- [Disentangling representations of retinal images with generative models](https://mlsanity.com/paper/2402.19186/) (2024)
- [Overlooked Implications of the Reconstruction Loss for VAE Disentanglement](https://mlsanity.com/paper/2202.13341/) (2022)
- [Gromov-Wasserstein Autoencoders](https://mlsanity.com/paper/2209.07007/) (2022)
- [Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability](https://mlsanity.com/paper/2512.13568/) (2025)
- [Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks](https://mlsanity.com/paper/2305.01713/) (2023)
- [Matching aggregate posteriors in the variational autoencoder](https://mlsanity.com/paper/2311.07693/) (2023)
- [Discovering the building blocks of dark matter halo density profiles with neural networks](https://mlsanity.com/paper/2203.08827/) (2022)
- [Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence](https://mlsanity.com/paper/2409.09413/) (2024)
- [Unsupervised Disentanglement with Tensor Product Representations on the Torus](https://mlsanity.com/paper/2202.06201/) (2022)
- [Physics-informed Variational Autoencoders for Improved Robustness to Environmental Factors of Variation](https://mlsanity.com/paper/2210.10418/) (2022)
- [Disentanglement Learning via Topology](https://mlsanity.com/paper/2308.12696/) (2023)
- [SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting](https://mlsanity.com/paper/2412.14371/) (2024)

## Abstract (arXiv)

Learning to disentangle and represent factors of variation in data is an important problem in AI. While many advances have been made to learn these representations, it is still unclear how to quantify disentanglement. While several metrics exist, little is known on their implicit assumptions, what they truly measure, and their limits. In consequence, it is difficult to interpret results when comparing different representations. In this work, we survey supervised disentanglement metrics and thoroughly analyze them. We propose a new taxonomy in which all metrics fall into one of three families: intervention-based, predictor-based and information-based. We conduct extensive experiments in which we isolate properties of disentangled representations, allowing stratified comparison along several axes. From our experiment results and analysis, we provide insights on relations between disentangled representation properties. Finally, we share guidelines on how to measure disentanglement.