# Theory and Evaluation Metrics for Learning Disentangled Representations

arXiv: https://arxiv.org/abs/1908.09961 · HTML: https://mlsanity.com/paper/1908.09961/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2019.

## Built on by (most cited first, 30 of 57)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Vector-Decomposed Disentanglement for Domain-Invariant Object Detection](https://mlsanity.com/paper/2108.06685/) (2021)
- [Self-Supervised Learning Disentangled Group Representation as Feature](https://mlsanity.com/paper/2110.15255/) (2021)
- [Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment](https://mlsanity.com/paper/2305.12218/) (2023)
- [Unsupervised Skill Discovery with Bottleneck Option Learning](https://mlsanity.com/paper/2106.14305/) (2021)
- [Where and What? Examining Interpretable Disentangled Representations](https://mlsanity.com/paper/2104.05622/) (2021)
- [Commutative Lie Group VAE for Disentanglement Learning](https://mlsanity.com/paper/2106.03375/) (2021)
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](https://mlsanity.com/paper/2302.02913/) (2023)
- [Non-negative Contrastive Learning](https://mlsanity.com/paper/2403.12459/) (2024)
- [Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximization](https://mlsanity.com/paper/2110.04686/) (2021)
- [Evaluating Readability and Faithfulness of Concept-based Explanations](https://mlsanity.com/paper/2404.18533/) (2024)
- [Disentanglement Analysis with Partial Information Decomposition](https://mlsanity.com/paper/2108.13753/) (2021)
- [Review of Disentanglement Approaches for Medical Applications -- Towards Solving the Gordian Knot of Generative Models in Healthcare](https://mlsanity.com/paper/2203.11132/) (2022)
- [Gromov-Wasserstein Autoencoders](https://mlsanity.com/paper/2209.07007/) (2022)
- [Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks](https://mlsanity.com/paper/2505.06224/) (2025)
- [Explaining, Evaluating and Enhancing Neural Networks' Learned Representations](https://mlsanity.com/paper/2202.09374/) (2022)
- [Deep Kernel Principal Component Analysis for Multi-level Feature Learning](https://mlsanity.com/paper/2302.11220/) (2023)
- [DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection](https://mlsanity.com/paper/2304.03323/) (2023)
- [Leveraging Relational Information for Learning Weakly Disentangled Representations](https://mlsanity.com/paper/2205.10056/) (2022)
- [Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2506.10629/) (2025)
- [On the relationship between disentanglement and multi-task learning](https://mlsanity.com/paper/2110.03498/) (2021)
- [Unsupervised Disentanglement with Tensor Product Representations on the Torus](https://mlsanity.com/paper/2202.06201/) (2022)
- [Distortion-Disentangled Contrastive Learning](https://mlsanity.com/paper/2303.05066/) (2023)
- [α-TCVAE: On the relationship between Disentanglement and Diversity](https://mlsanity.com/paper/2411.00588/) (2024)
- [Disentangling Autoencoders (DAE)](https://mlsanity.com/paper/2202.09926/) (2022)
- [A Category-theoretical Meta-analysis of Definitions of Disentanglement](https://mlsanity.com/paper/2305.06886/) (2023)
- [Evaluation of Correctness in Unsupervised Many-to-Many Image Translation](https://mlsanity.com/paper/2103.15727/) (2021)
- [Correcting Flaws in Common Disentanglement Metrics](https://mlsanity.com/paper/2304.02335/) (2023)
- [Enriching Disentanglement: From Logical Definitions to Quantitative Metrics](https://mlsanity.com/paper/2305.11512/) (2023)

## Abstract (arXiv)

We make two theoretical contributions to disentanglement learning by (a) defining precise semantics of disentangled representations, and (b) establishing robust metrics for evaluation. First, we characterize the concept "disentangled representations" used in supervised and unsupervised methods along three dimensions-informativeness, separability and interpretability - which can be expressed and quantified explicitly using information-theoretic constructs. This helps explain the behaviors of several well-known disentanglement learning models. We then propose robust metrics for measuring informativeness, separability and interpretability. Through a comprehensive suite of experiments, we show that our metrics correctly characterize the representations learned by different methods and are consistent with qualitative (visual) results. Thus, the metrics allow disentanglement learning methods to be compared on a fair ground. We also empirically uncovered new interesting properties of VAE-based methods and interpreted them with our formulation. These findings are promising and hopefully will encourage the design of more theoretically driven models for learning disentangled representations.