# Closing the Dequantization Gap: PixelCNN as a Single-Layer Flow

arXiv: https://arxiv.org/abs/2002.02547 · HTML: https://mlsanity.com/paper/2002.02547/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flows](https://mlsanity.com/paper/2202.03281/) (2022)
- [Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization](https://mlsanity.com/paper/2202.11453/) (2022)
- [Simulating 2+1D Lattice Quantum Electrodynamics at Finite Density with Neural Flow Wavefunctions](https://mlsanity.com/paper/2212.06835/) (2022)
- [Understanding Likelihood of Normalizing Flow and Image Complexity through the Lens of Out-of-Distribution Detection](https://mlsanity.com/paper/2402.10477/) (2024)
- [Multi-Resolution Continuous Normalizing Flows](https://mlsanity.com/paper/2106.08462/) (2021)
- [Learning to Bound: A Generative Cram'er-Rao Bound](https://mlsanity.com/paper/2203.03695/) (2022)

## Abstract (arXiv)

Flow models have recently made great progress at modeling ordinal discrete data such as images and audio. Due to the continuous nature of flow models, dequantization is typically applied when using them for such discrete data, resulting in lower bound estimates of the likelihood. In this paper, we introduce subset flows, a class of flows that can tractably transform finite volumes and thus allow exact computation of likelihoods for discrete data. Based on subset flows, we identify ordinal discrete autoregressive models, including WaveNets, PixelCNNs and Transformers, as single-layer flows. We use the flow formulation to compare models trained and evaluated with either the exact likelihood or its dequantization lower bound. Finally, we study multilayer flows composed of PixelCNNs and non-autoregressive coupling layers and demonstrate state-of-the-art results on CIFAR-10 for flow models trained with dequantization.