# Canonical normalizing flows for manifold learning

arXiv: https://arxiv.org/abs/2310.12743 · HTML: https://mlsanity.com/paper/2310.12743/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Normalizing Flows on Tori and Spheres](https://mlsanity.com/paper/2002.02428/) (2020)
- [Rectangular Flows for Manifold Learning](https://mlsanity.com/paper/2106.01413/) (2021)
- [Learning Flat Latent Manifolds with VAEs](https://mlsanity.com/paper/2002.04881/) (2020)
- [TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization](https://mlsanity.com/paper/2303.05506/) (2023)
- [Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows](https://mlsanity.com/paper/2106.05275/) (2021)
- [Variational Relevance Vector Machines](https://mlsanity.com/paper/1301.3838/) (2013)
- [Principal Component Density Estimation for Scenario Generation Using Normalizing Flows](https://mlsanity.com/paper/2104.10410/) (2021)
- [Representing Deep Neural Networks Latent Space Geometries with Graphs](https://mlsanity.com/paper/2011.07343/) (2020)
- [Sparse PCA from Sparse Linear Regression](https://mlsanity.com/paper/1811.10106/) (2018)
- [Nonlinear Isometric Manifold Learning for Injective Normalizing Flows](https://mlsanity.com/paper/2203.03934/) (2022)

## Built on by (most cited first, 15 of 15)

- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [A Matrix Product State Model for Simultaneous Classification and Generation](https://mlsanity.com/paper/2406.17441/) (2024)
- [Manifold Learning with Normalizing Flows: Towards Regularity, Expressivity and Iso-Riemannian Geometry](https://mlsanity.com/paper/2505.08087/) (2025)
- [Explicit and data-Efficient Encoding via Gradient Flow](https://mlsanity.com/paper/2412.00864/) (2024)
- [Localized FNO for Spatiotemporal Hemodynamic Upsampling in Aneurysm MRI](https://mlsanity.com/paper/2507.13789/) (2025)
- [Injective flows for star-like manifolds](https://mlsanity.com/paper/2406.09116/) (2024)
- [Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models](https://mlsanity.com/paper/2407.08843/) (2024)
- [Energy-Based Prior Latent Space Diffusion model for Reconstruction of Lumbar Vertebrae from Thick Slice MRI](https://mlsanity.com/paper/2412.00511/) (2024)
- [Estimating Dataset Dimension via Singular Metrics under the Manifold Hypothesis: Application to Inverse Problems](https://mlsanity.com/paper/2507.07291/) (2025)
- [Operator Flow Matching for Timeseries Forecasting](https://mlsanity.com/paper/2510.15101/) (2025)
- [Multimodal Scientific Learning Beyond Diffusions and Flows](https://mlsanity.com/paper/2602.00960/) (2026)
- [FloDR: An invertible dimensionality reduction method based on a normalising flow](https://mlsanity.com/paper/2607.26278/) (2026)
- [Hyperbolic Optimization](https://mlsanity.com/paper/2509.25206/) (2025)
- [Recovering Weak Signals with Normalizing Flows](https://mlsanity.com/paper/2609.06382/) (2026)
- [Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting](https://mlsanity.com/paper/2609.15344/) (2026)

## Abstract (arXiv)

Manifold learning flows are a class of generative modelling techniques that assume a low-dimensional manifold description of the data. The embedding of such a manifold into the high-dimensional space of the data is achieved via learnable invertible transformations. Therefore, once the manifold is properly aligned via a reconstruction loss, the probability density is tractable on the manifold and maximum likelihood can be used to optimize the network parameters. Naturally, the lower-dimensional representation of the data requires an injective-mapping. Recent approaches were able to enforce that the density aligns with the modelled manifold, while efficiently calculating the density volume-change term when embedding to the higher-dimensional space. However, unless the injective-mapping is analytically predefined, the learned manifold is not necessarily an efficient representation of the data. Namely, the latent dimensions of such models frequently learn an entangled intrinsic basis, with degenerate information being stored in each dimension. Alternatively, if a locally orthogonal and/or sparse basis is to be learned, here coined canonical intrinsic basis, it can serve in learning a more compact latent space representation. Toward this end, we propose a canonical manifold learning flow method, where a novel optimization objective enforces the transformation matrix to have few prominent and non-degenerate basis functions. We demonstrate that by minimizing the off-diagonal manifold metric elements ℓ1-norm, we can achieve such a basis, which is simultaneously sparse and/or orthogonal. Canonical manifold flow yields a more efficient use of the latent space, automatically generating fewer prominent and distinct dimensions to represent data, and a better approximation of target distributions than other manifold flow methods in most experiments we conducted, resulting in lower FID scores.