# Neural Autoregressive Flows

arXiv: https://arxiv.org/abs/1804.00779 · HTML: https://mlsanity.com/paper/1804.00779/

## Key facts

- Cited by 208 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2018.

## Built on by (most cited first, 30 of 208)

- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [A Variational Perspective on Diffusion-Based Generative Models and Score Matching](https://mlsanity.com/paper/2106.02808/) (2021)
- [Riemannian Diffusion Models](https://mlsanity.com/paper/2208.07949/) (2022)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Efficient Neural Causal Discovery without Acyclicity Constraints](https://mlsanity.com/paper/2107.10483/) (2021)
- [Diffusion Normalizing Flow](https://mlsanity.com/paper/2110.07579/) (2021)
- [Flow-based Kernel Prior with Application to Blind Super-Resolution](https://mlsanity.com/paper/2103.15977/) (2021)
- [Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling](https://mlsanity.com/paper/2108.05301/) (2021)
- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Smooth Normalizing Flows](https://mlsanity.com/paper/2110.00351/) (2021)
- [Copula Flows for Synthetic Data Generation](https://mlsanity.com/paper/2101.00598/) (2021)
- [A Survey on Causal Discovery Methods for I.I.D. and Time Series Data](https://mlsanity.com/paper/2303.15027/) (2023)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Autoregressive Conditional Neural Processes](https://mlsanity.com/paper/2303.14468/) (2023)
- [Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint](https://mlsanity.com/paper/2109.11375/) (2021)
- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [Implicit Normalizing Flows](https://mlsanity.com/paper/2103.09527/) (2021)
- [iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder](https://mlsanity.com/paper/2111.00965/) (2021)
- [TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series](https://mlsanity.com/paper/2310.01327/) (2023)
- [Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation](https://mlsanity.com/paper/2211.01939/) (2022)
- [Generalized Normalizing Flows via Markov Chains](https://mlsanity.com/paper/2111.12506/) (2021)
- [Local-Global MCMC kernels: the best of both worlds](https://mlsanity.com/paper/2111.02702/) (2021)
- [Distribution learning via neural differential equations: a nonparametric statistical perspective](https://mlsanity.com/paper/2309.01043/) (2023)
- [FloMo: Tractable Motion Prediction with Normalizing Flows](https://mlsanity.com/paper/2103.03614/) (2021)

## Abstract (arXiv)

Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify and generalize these approaches, replacing the (conditionally) affine univariate transformations of MAF/IAF with a more general class of invertible univariate transformations expressed as monotonic neural networks. We demonstrate that the proposed neural autoregressive flows (NAF) are universal approximators for continuous probability distributions, and their greater expressivity allows them to better capture multimodal target distributions. Experimentally, NAF yields state-of-the-art performance on a suite of density estimation tasks and outperforms IAF in variational autoencoders trained on binarized MNIST.