# Disentanglement Analysis with Partial Information Decomposition

arXiv: https://arxiv.org/abs/2108.13753 · HTML: https://mlsanity.com/paper/2108.13753/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 31 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Learning Disentangled Joint Continuous and Discrete Representations](https://mlsanity.com/paper/1804.00104/) (2018)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [Theory and Evaluation Metrics for Learning Disentangled Representations](https://mlsanity.com/paper/1908.09961/) (2019)
- [Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration](https://mlsanity.com/paper/1804.06537/) (2018)
- [Progressive Learning and Disentanglement of Hierarchical Representations](https://mlsanity.com/paper/2002.10549/) (2020)
- [Auto-Encoding Total Correlation Explanation](https://mlsanity.com/paper/1802.05822/) (2018)
- [Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems](https://mlsanity.com/paper/1411.2832/) (2014)
- [Unsupervised Robust Disentangling of Latent Characteristics for Image Synthesis](https://mlsanity.com/paper/1910.10223/) (2019)

## Built on by (most cited first, 9 of 9)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [A Measure of the Complexity of Neural Representations based on Partial Information Decomposition](https://mlsanity.com/paper/2209.10438/) (2022)
- [Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination](https://mlsanity.com/paper/2311.06673/) (2023)
- [A Category-theoretical Meta-analysis of Definitions of Disentanglement](https://mlsanity.com/paper/2305.06886/) (2023)
- [What should a neuron aim for? Designing local objective functions based on information theory](https://mlsanity.com/paper/2412.02482/) (2024)
- [Enriching Disentanglement: From Logical Definitions to Quantitative Metrics](https://mlsanity.com/paper/2305.11512/) (2023)
- [Analyzing Generative Models by Manifold Entropic Metrics](https://mlsanity.com/paper/2410.19426/) (2024)
- [Equivariant Disentangled Transformation for Domain Generalization under Combination Shift](https://mlsanity.com/paper/2208.02011/) (2022)
- [Synergistic Information Disentanglement for Omni-modal Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2609.02118/) (2026)

## Abstract (arXiv)

We propose a framework to analyze how multivariate representations disentangle ground-truth generative factors. A quantitative analysis of disentanglement has been based on metrics designed to compare how one variable explains each generative factor. Current metrics, however, may fail to detect entanglement that involves more than two variables, e.g., representations that duplicate and rotate generative factors in high dimensional spaces. In this work, we establish a framework to analyze information sharing in a multivariate representation with Partial Information Decomposition and propose a new disentanglement metric. This framework enables us to understand disentanglement in terms of uniqueness, redundancy, and synergy. We develop an experimental protocol to assess how increasingly entangled representations are evaluated with each metric and confirm that the proposed metric correctly responds to entanglement. Through experiments on variational autoencoders, we find that models with similar disentanglement scores have a variety of characteristics in entanglement, for each of which a distinct strategy may be required to obtain a disentangled representation.