# From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows

arXiv: https://arxiv.org/abs/2602.06940 · HTML: https://mlsanity.com/paper/2602.06940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Builds on (most cited first, 27 of 27)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [InfoVAE: Information Maximizing Variational Autoencoders](https://mlsanity.com/paper/1706.02262/) (2017)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)](https://mlsanity.com/paper/2001.04872/) (2020)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Learning Ordered Representations with Nested Dropout](https://mlsanity.com/paper/1402.0915/) (2014)
- [Learning Disentangled Joint Continuous and Discrete Representations](https://mlsanity.com/paper/1804.00104/) (2018)
- [Function Classes for Identifiable Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2208.06406/) (2022)
- [Rectangular Flows for Manifold Learning](https://mlsanity.com/paper/2106.01413/) (2021)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis](https://mlsanity.com/paper/2506.06276/) (2025)
- [Diagnosing and Fixing Manifold Overfitting in Deep Generative Models](https://mlsanity.com/paper/2204.07172/) (2022)
- [Embrace the Gap: VAEs Perform Independent Mechanism Analysis](https://mlsanity.com/paper/2206.02416/) (2022)
- [Density estimation on low-dimensional manifolds: an inflation-deflation approach](https://mlsanity.com/paper/2105.12152/) (2021)
- [Ordering Dimensions with Nested Dropout Normalizing Flows](https://mlsanity.com/paper/2006.08777/) (2020)
- [Independent Mechanism Analysis and the Manifold Hypothesis](https://mlsanity.com/paper/2312.13438/) (2023)
- [Denoising Deep Generative Models](https://mlsanity.com/paper/2212.01265/) (2022)
- [Analyzing Generative Models by Manifold Entropic Metrics](https://mlsanity.com/paper/2410.19426/) (2024)
- [Beyond Diagonal Covariance: Flexible Posterior VAEs via Free-Form Injective Flows](https://mlsanity.com/paper/2506.01522/) (2025)

## Abstract (arXiv)

The unsupervised discovery of features that are both semantically meaningful and stable across runs remains a central challenge in representation learning. We introduce entropy-ordered flows (EOFlows), a normalizing flow (NF) framework that augments standard maximum likelihood training with an orthogonality regularizer on the decoder Jacobian. The regularizer is rooted in Independent Mechanism Analysis and encourages geometric disentanglement, and a stochastic estimator makes it tractable at image scale (CelebA at D=2352 and 12288). Learned features form near-orthogonal curvilinear coordinates and can be ordered by their textitexplained (manifold) entropy after training, analogous to the ranking by explained variance in PCA, which turns EOFlows into a non-linear generalization of PCA. EOFlows identify an order of magnitude more stable features than existing methods, and these features emerge in distinguishable categories (global, local, and generic) and support tentative semantic interpretations. The local features have strikingly sparse support in pixel space, although our method never enforces this. Retaining only the most important, i.e. highest entropy, features turns the bijective flow into an autoencoder with adjustable bottleneck, rivaling the rate-distortion performance of dedicated autoencoders.