# Principal Manifold Flows

arXiv: https://arxiv.org/abs/2202.07037 · HTML: https://mlsanity.com/paper/2202.07037/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Normalizing Flows on Tori and Spheres](https://mlsanity.com/paper/2002.02428/) (2020)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization](https://mlsanity.com/paper/2012.05942/) (2020)
- [NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport](https://mlsanity.com/paper/1903.03704/) (2019)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Normalizing Flows on Riemannian Manifolds](https://mlsanity.com/paper/1611.02304/) (2016)
- [Rectangular Flows for Manifold Learning](https://mlsanity.com/paper/2106.01413/) (2021)
- [Latent Variable Modelling with Hyperbolic Normalizing Flows](https://mlsanity.com/paper/2002.06336/) (2020)
- [Trumpets: Injective Flows for Inference and Inverse Problems](https://mlsanity.com/paper/2102.10461/) (2021)
- [Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation](https://mlsanity.com/paper/2108.07668/) (2021)
- [Regularized Autoencoders via Relaxed Injective Probability Flow](https://mlsanity.com/paper/2002.08927/) (2020)
- [Normalizing Flows Across Dimensions](https://mlsanity.com/paper/2006.13070/) (2020)
- [Isometric Autoencoders](https://mlsanity.com/paper/2006.09289/) (2020)
- [Principal Component Density Estimation for Scenario Generation Using Normalizing Flows](https://mlsanity.com/paper/2104.10410/) (2021)
- [Generative Locally Linear Embedding](https://mlsanity.com/paper/2104.01525/) (2021)

## Built on by (most cited first, 4 of 4)

- [Verifying the Union of Manifolds Hypothesis for Image Data](https://mlsanity.com/paper/2207.02862/) (2022)
- [Independent Mechanism Analysis and the Manifold Hypothesis](https://mlsanity.com/paper/2312.13438/) (2023)
- [Nonlinear Isometric Manifold Learning for Injective Normalizing Flows](https://mlsanity.com/paper/2203.03934/) (2022)
- [Probing the Robustness of Independent Mechanism Analysis for Representation Learning](https://mlsanity.com/paper/2207.06137/) (2022)

## Abstract (arXiv)

Normalizing flows map an independent set of latent variables to their samples using a bijective transformation. Despite the exact correspondence between samples and latent variables, their high level relationship is not well understood. In this paper we characterize the geometric structure of flows using principal manifolds and understand the relationship between latent variables and samples using contours. We introduce a novel class of normalizing flows, called principal manifold flows (PF), whose contours are its principal manifolds, and a variant for injective flows (iPF) that is more efficient to train than regular injective flows. PFs can be constructed using any flow architecture, are trained with a regularized maximum likelihood objective and can perform density estimation on all of their principal manifolds. In our experiments we show that PFs and iPFs are able to learn the principal manifolds over a variety of datasets. Additionally, we show that PFs can perform density estimation on data that lie on a manifold with variable dimensionality, which is not possible with existing normalizing flows.