# Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows

arXiv: https://arxiv.org/abs/2110.04227 · HTML: https://mlsanity.com/paper/2110.04227/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [The Reversible Residual Network: Backpropagation Without Storing Activations](https://mlsanity.com/paper/1707.04585/) (2017)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [i-RevNet: Deep Invertible Networks](https://mlsanity.com/paper/1802.07088/) (2018)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Reverse-Engineering Deep ReLU Networks](https://mlsanity.com/paper/1910.00744/) (2019)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging](https://mlsanity.com/paper/2010.14462/) (2020)
- [Trumpets: Injective Flows for Inference and Inverse Problems](https://mlsanity.com/paper/2102.10461/) (2021)
- [Inverting Deep Generative models, One layer at a time](https://mlsanity.com/paper/1906.07437/) (2019)
- [HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference](https://mlsanity.com/paper/1905.10687/) (2019)
- [On the ability of neural nets to express distributions](https://mlsanity.com/paper/1702.07028/) (2017)
- [Benchmarking Invertible Architectures on Inverse Problems](https://mlsanity.com/paper/2101.10763/) (2021)
- [Globally Injective ReLU Networks](https://mlsanity.com/paper/2006.08464/) (2020)
- [Normalizing Flows Across Dimensions](https://mlsanity.com/paper/2006.13070/) (2020)
- [When and How Can Deep Generative Models be Inverted?](https://mlsanity.com/paper/2006.15555/) (2020)
- [A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions](https://mlsanity.com/paper/2004.08867/) (2020)

## Built on by (most cited first, 13 of 13)

- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [Conditional Injective Flows for Bayesian Imaging](https://mlsanity.com/paper/2204.07664/) (2022)
- [Globally injective and bijective neural operators](https://mlsanity.com/paper/2306.03982/) (2023)
- [Injectivity of ReLU networks: perspectives from statistical physics](https://mlsanity.com/paper/2302.14112/) (2023)
- [Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?](https://mlsanity.com/paper/2204.11231/) (2022)
- [Continuous Generative Neural Networks: A Wavelet-Based Architecture in Function Spaces](https://mlsanity.com/paper/2205.14627/) (2022)
- [Injectivity capacity of ReLU gates](https://mlsanity.com/paper/2410.20646/) (2024)
- [Deep Injective Prior for Inverse Scattering](https://mlsanity.com/paper/2301.03092/) (2023)
- [On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networks](https://mlsanity.com/paper/2308.09367/) (2023)
- [Deep Invertible Approximation of Topologically Rich Maps between Manifolds](https://mlsanity.com/paper/2210.00577/) (2022)
- [Injectivity of ReLU-layers: Tools from Frame Theory](https://mlsanity.com/paper/2406.15856/) (2024)
- [Kinks vs. Smoothness: Identifiability of Real Analytic nICA for Laplace-like Sources](https://mlsanity.com/paper/2609.21926/) (2026)

## Abstract (arXiv)

We study approximation of probability measures supported on n-dimensional manifolds embedded in ℝm by injective flows -- neural networks composed of invertible flows and injective layers. We show that in general, injective flows between ℝn and ℝm universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, m ≥ 3n+1, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer and Cranmer 2020.