# Auto-Encoding Total Correlation Explanation

arXiv: https://arxiv.org/abs/1802.05822 · HTML: https://mlsanity.com/paper/1802.05822/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2018.

## Built on by (most cited first, 17 of 17)

- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Lifelong Teacher-Student Network Learning](https://mlsanity.com/paper/2107.04689/) (2021)
- [Unsupervised Belief Representation Learning with Information-Theoretic Variational Graph Auto-Encoders](https://mlsanity.com/paper/2110.00210/) (2021)
- [Lifelong Mixture of Variational Autoencoders](https://mlsanity.com/paper/2107.04694/) (2021)
- [LADIS: Language Disentanglement for 3D Shape Editing](https://mlsanity.com/paper/2212.05011/) (2022)
- [Disentanglement Analysis with Partial Information Decomposition](https://mlsanity.com/paper/2108.13753/) (2021)
- [Semi-Supervised Disentanglement of Class-Related and Class-Independent Factors in VAE](https://mlsanity.com/paper/2102.00892/) (2021)
- [InfoVAEGAN : learning joint interpretable representations by information maximization and maximum likelihood](https://mlsanity.com/paper/2107.04705/) (2021)
- [Deep Neural Networks to Recover Unknown Physical Parameters from Oscillating Time Series](https://mlsanity.com/paper/2101.03850/) (2021)
- [Invariant Representations with Stochastically Quantized Neural Networks](https://mlsanity.com/paper/2208.02656/) (2022)
- [α-TCVAE: On the relationship between Disentanglement and Diversity](https://mlsanity.com/paper/2411.00588/) (2024)
- [PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization](https://mlsanity.com/paper/2507.16679/) (2025)
- [C2VAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior](https://mlsanity.com/paper/2309.13303/) (2023)
- [On the Separability of Information in Diffusion Models](https://mlsanity.com/paper/2509.23937/) (2025)
- [TC-VAE: Uncovering Out-of-Distribution Data Generative Factors](https://mlsanity.com/paper/2304.04103/) (2023)
- [IROTE: Human-like Traits Elicitation of Large Language Model via In-Context Self-Reflective Optimization](https://mlsanity.com/paper/2508.08719/) (2025)
- [Maximum Total Correlation Reinforcement Learning](https://mlsanity.com/paper/2505.16734/) (2025)

## Abstract (arXiv)

Advances in unsupervised learning enable reconstruction and generation of samples from complex distributions, but this success is marred by the inscrutability of the representations learned. We propose an information-theoretic approach to characterizing disentanglement and dependence in representation learning using multivariate mutual information, also called total correlation. The principle of total Cor-relation Ex-planation (CorEx) has motivated successful unsupervised learning applications across a variety of domains, but under some restrictive assumptions. Here we relax those restrictions by introducing a flexible variational lower bound to CorEx. Surprisingly, we find that this lower bound is equivalent to the one in variational autoencoders (VAE) under certain conditions. This information-theoretic view of VAE deepens our understanding of hierarchical VAE and motivates a new algorithm, AnchorVAE, that makes latent codes more interpretable through information maximization and enables generation of richer and more realistic samples.