# Universal Approximation of Residual Flows in Maximum Mean Discrepancy

arXiv: https://arxiv.org/abs/2103.05793 · HTML: https://mlsanity.com/paper/2103.05793/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [Representation Benefits of Deep Feedforward Networks](https://mlsanity.com/paper/1509.08101/) (2015)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [ResNet with one-neuron hidden layers is a Universal Approximator](https://mlsanity.com/paper/1806.10909/) (2018)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [The Expressive Power of a Class of Normalizing Flow Models](https://mlsanity.com/paper/2006.00392/) (2020)
- [Approximating Lipschitz continuous functions with GroupSort neural networks](https://mlsanity.com/paper/2006.05254/) (2020)
- [On the ability of neural nets to express distributions](https://mlsanity.com/paper/1702.07028/) (2017)
- [Representational aspects of depth and conditioning in normalizing flows](https://mlsanity.com/paper/2010.01155/) (2020)
- [Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks](https://mlsanity.com/paper/2006.16664/) (2020)
- [Size-Noise Tradeoffs in Generative Networks](https://mlsanity.com/paper/1810.11158/) (2018)
- [Universal Lipschitz Approximation in Bounded Depth Neural Networks](https://mlsanity.com/paper/1904.04861/) (2019)
- [A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions](https://mlsanity.com/paper/2004.08867/) (2020)
- [On the space-time expressivity of ResNets](https://mlsanity.com/paper/1910.09599/) (2019)

## Built on by (most cited first, 7 of 7)

- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [On the expressivity of bi-Lipschitz normalizing flows](https://mlsanity.com/paper/2107.07232/) (2021)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generators](https://mlsanity.com/paper/2506.01158/) (2025)
- [Learning to Bound: A Generative Cram'er-Rao Bound](https://mlsanity.com/paper/2203.03695/) (2022)
- [Arbitrary Distributions Mapping via SyMOT-Flow: A Flow-based Approach Integrating Maximum Mean Discrepancy and Optimal Transport](https://mlsanity.com/paper/2308.13815/) (2023)
- [Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspective](https://mlsanity.com/paper/2605.06172/) (2026)

## Abstract (arXiv)

Normalizing flows are a class of flexible deep generative models that offer easy likelihood computation. Despite their empirical success, there is little theoretical understanding of their expressiveness. In this work, we study residual flows, a class of normalizing flows composed of Lipschitz residual blocks. We prove residual flows are universal approximators in maximum mean discrepancy. We provide upper bounds on the number of residual blocks to achieve approximation under different assumptions.