# FInC Flow: Fast and Invertible k × k Convolutions for Normalizing Flows

arXiv: https://arxiv.org/abs/2301.09266 · HTML: https://mlsanity.com/paper/2301.09266/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 23 Jan 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [PolyGen: An Autoregressive Generative Model of 3D Meshes](https://mlsanity.com/paper/2002.10880/) (2020)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Generative Flow Networks for Discrete Probabilistic Modeling](https://mlsanity.com/paper/2202.01361/) (2022)
- [MintNet: Building Invertible Neural Networks with Masked Convolutions](https://mlsanity.com/paper/1907.07945/) (2019)
- [MaCow: Masked Convolutional Generative Flow](https://mlsanity.com/paper/1902.04208/) (2019)
- [Emerging Convolutions for Generative Normalizing Flows](https://mlsanity.com/paper/1901.11137/) (2019)
- [Efficient-VDVAE: Less is more](https://mlsanity.com/paper/2203.13751/) (2022)
- [Self Normalizing Flows](https://mlsanity.com/paper/2011.07248/) (2020)
- [ButterflyFlow: Building Invertible Layers with Butterfly Matrices](https://mlsanity.com/paper/2209.13774/) (2022)
- [Woodbury Transformations for Deep Generative Flows](https://mlsanity.com/paper/2002.12229/) (2020)

## Built on by (most cited first, 2 of 2)

- [Adaptation of the super resolution SOTA for Art Restoration in camera capture images](https://mlsanity.com/paper/2309.13655/) (2023)
- [Parallel Backpropagation for Inverse of a Convolution with Application to Normalizing Flows](https://mlsanity.com/paper/2410.14634/) (2024)

## Abstract (arXiv)

Invertible convolutions have been an essential element for building expressive normalizing flow-based generative models since their introduction in Glow. Several attempts have been made to design invertible k × k convolutions that are efficient in training and sampling passes. Though these attempts have improved the expressivity and sampling efficiency, they severely lagged behind Glow which used only 1 × 1 convolutions in terms of sampling time. Also, many of the approaches mask a large number of parameters of the underlying convolution, resulting in lower expressivity on a fixed run-time budget. We propose a k × k convolutional layer and Deep Normalizing Flow architecture which i.) has a fast parallel inversion algorithm with running time O(n k2) (n is height and width of the input image and k is kernel size), ii.) masks the minimal amount of learnable parameters in a layer. iii.) gives better forward pass and sampling times comparable to other k × k convolution-based models on real-world benchmarks. We provide an implementation of the proposed parallel algorithm for sampling using our invertible convolutions on GPUs. Benchmarks on CIFAR-10, ImageNet, and CelebA datasets show comparable performance to previous works regarding bits per dimension while significantly improving the sampling time.