# VFlow: More Expressive Generative Flows with Variational Data Augmentation

arXiv: https://arxiv.org/abs/2002.09741 · HTML: https://mlsanity.com/paper/2002.09741/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2020.

## Built on by (most cited first, 30 of 38)

- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theory](https://mlsanity.com/paper/2110.11291/) (2021)
- [Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics](https://mlsanity.com/paper/2302.00600/) (2023)
- [Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamics](https://mlsanity.com/paper/2302.01170/) (2023)
- [Densely connected normalizing flows](https://mlsanity.com/paper/2106.04627/) (2021)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Maximum Likelihood Training of Implicit Nonlinear Diffusion Models](https://mlsanity.com/paper/2205.13699/) (2022)
- [Flow-matching -- efficient coarse-graining of molecular dynamics without forces](https://mlsanity.com/paper/2203.11167/) (2022)
- [Implicit Normalizing Flows](https://mlsanity.com/paper/2103.09527/) (2021)
- [Rigid Body Flows for Sampling Molecular Crystal Structures](https://mlsanity.com/paper/2301.11355/) (2023)
- [Fast Inference and Update of Probabilistic Density Estimation on Trajectory Prediction](https://mlsanity.com/paper/2308.08824/) (2023)
- [iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder](https://mlsanity.com/paper/2111.00965/) (2021)
- [PD-Flow: A Point Cloud Denoising Framework with Normalizing Flows](https://mlsanity.com/paper/2203.05940/) (2022)
- [Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model](https://mlsanity.com/paper/2306.01424/) (2023)
- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [ButterflyFlow: Building Invertible Layers with Butterfly Matrices](https://mlsanity.com/paper/2209.13774/) (2022)
- [PassFlow: Guessing Passwords with Generative Flows](https://mlsanity.com/paper/2105.06165/) (2021)
- [Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flows](https://mlsanity.com/paper/2107.02951/) (2021)
- [Hamiltonian Score Matching and Generative Flows](https://mlsanity.com/paper/2410.20470/) (2024)
- [Invertible Monotone Operators for Normalizing Flows](https://mlsanity.com/paper/2210.08176/) (2022)
- [ELSA -- Enhanced latent spaces for improved collider simulations](https://mlsanity.com/paper/2305.07696/) (2023)
- [Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Models](https://mlsanity.com/paper/2407.04958/) (2024)
- [Multi-Resolution Continuous Normalizing Flows](https://mlsanity.com/paper/2106.08462/) (2021)
- [Funnels: Exact maximum likelihood with dimensionality reduction](https://mlsanity.com/paper/2112.08069/) (2021)
- [Fast Lossless Neural Compression with Integer-Only Discrete Flows](https://mlsanity.com/paper/2206.08869/) (2022)
- [A Non-negative VAE:the Generalized Gamma Belief Network](https://mlsanity.com/paper/2408.03388/) (2024)
- [Taming Normalizing Flows](https://mlsanity.com/paper/2211.16488/) (2022)

## Abstract (arXiv)

Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations. However, tractability imposes architectural constraints on generative flows, making them less expressive than other types of generative models. In this work, we study a previously overlooked constraint that all the intermediate representations must have the same dimensionality with the original data due to invertibility, limiting the width of the network. We tackle this constraint by augmenting the data with some extra dimensions and jointly learning a generative flow for augmented data as well as the distribution of augmented dimensions under a variational inference framework. Our approach, VFlow, is a generalization of generative flows and therefore always performs better. Combining with existing generative flows, VFlow achieves a new state-of-the-art 2.98 bits per dimension on the CIFAR-10 dataset and is more compact than previous models to reach similar modeling quality.