ELF: Exact-Lipschitz Based Universal Density Approximator Flow
Not on the map. Why?
Abstract (arXiv)
Normalizing flows have grown more popular over the last few years; however, they continue to be computationally expensive, making them difficult to be accepted into the broader machine learning community. In this paper, we introduce a simple one-dimensional one-layer network that has closed form Lipschitz constants; using this, we introduce a new Exact-Lipschitz Flow (ELF) that combines the ease of sampling from residual flows with the strong performance of autoregressive flows. Further, we show that ELF is provably a universal density approximator, more computationally and parameter efficient compared to a multitude of other flows, and achieves state-of-the-art performance on multiple large-scale datasets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 13 Dec 2021.
Builds on
39 of 51 references.
References · 39
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2013Intriguing properties of neural networkscited by 5,114
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2016Density estimation using Real NVPcited by 2,029
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
- 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
- 2014NICE: Non-linear Independent Components Estimationcited by 1,052
- 2016HyperNetworkscited by 1,003
27 more
- 2016Pixel Recurrent Neural Networkscited by 814
- 2017Masked Autoregressive Flow for Density Estimationcited by 680
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
- 2019Neural Spline Flowscited by 590
- 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
- 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
- 2018Invertible Residual Networkscited by 364
- 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
- 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
- 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
- 2019Residual Flows for Invertible Generative Modelingcited by 256
- 2018Sorting out Lipschitz function approximationcited by 211
- 2018Neural Autoregressive Flowscited by 207
- 2020Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimizationcited by 89
- 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
- 2018Sylvester Normalizing Flows for Variational Inferencecited by 71
- 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
- 2019Block Neural Autoregressive Flowcited by 62
- 2019Sum-of-Squares Polynomial Flowcited by 62
- 2020Accelerating Feedforward Computation via Parallel Nonlinear Equation Solvingcited by 51
- 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 48
- 2020VFlow: More Expressive Generative Flows with Variational Data Augmentationcited by 38
- 2020The Expressive Power of a Class of Normalizing Flow Modelscited by 35
- 2019MaCow: Masked Convolutional Generative Flowcited by 32
- 2018Transformation Autoregressive Networkscited by 27
- 2020Predictive Sampling with Forecasting Autoregressive Modelscited by 10
Built on by
Citing papers per year: 2022: 1.
1 paper.