# Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design

arXiv: https://arxiv.org/abs/1902.00275 · HTML: https://mlsanity.com/paper/1902.00275/

## Key facts

- Cited by 262 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2019.

## Built on by (most cited first, 30 of 262)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [Concrete Score Matching: Generalized Score Matching for Discrete Data](https://mlsanity.com/paper/2211.00802/) (2022)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Score-Based Diffusion Models as Principled Priors for Inverse Imaging](https://mlsanity.com/paper/2304.11751/) (2023)
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](https://mlsanity.com/paper/2103.16877/) (2021)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [Let us Build Bridges: Understanding and Extending Diffusion Generative Models](https://mlsanity.com/paper/2208.14699/) (2022)
- [Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation](https://mlsanity.com/paper/2106.05527/) (2021)
- [Poisson Flow Generative Models](https://mlsanity.com/paper/2209.11178/) (2022)
- [Invertible Image Signal Processing](https://mlsanity.com/paper/2103.15061/) (2021)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [Reflected Diffusion Models](https://mlsanity.com/paper/2304.04740/) (2023)
- [Diffusion Normalizing Flow](https://mlsanity.com/paper/2110.07579/) (2021)
- [Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling](https://mlsanity.com/paper/2108.05301/) (2021)
- [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022)
- [Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms](https://mlsanity.com/paper/2212.12678/) (2022)
- [Hierarchical VAEs Know What They Don't Know](https://mlsanity.com/paper/2102.08248/) (2021)
- [Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs](https://mlsanity.com/paper/2305.03935/) (2023)
- [Smooth Normalizing Flows](https://mlsanity.com/paper/2110.00351/) (2021)
- [Soft then Hard: Rethinking the Quantization in Neural Image Compression](https://mlsanity.com/paper/2104.05168/) (2021)

## Abstract (arXiv)

Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to state-of-the-art autoregressive models. In this paper, we investigate and improve upon three limiting design choices employed by flow-based models in prior work: the use of uniform noise for dequantization, the use of inexpressive affine flows, and the use of purely convolutional conditioning networks in coupling layers. Based on our findings, we propose Flow++, a new flow-based model that is now the state-of-the-art non-autoregressive model for unconditional density estimation on standard image benchmarks. Our work has begun to close the significant performance gap that has so far existed between autoregressive models and flow-based models. Our implementation is available at https://github.com/aravindsrinivas/flowpp