# Self Normalizing Flows

arXiv: https://arxiv.org/abs/2011.07248 · HTML: https://mlsanity.com/paper/2011.07248/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Free-form Flows: Make Any Architecture a Normalizing Flow](https://mlsanity.com/paper/2310.16624/) (2023)
- [Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flow](https://mlsanity.com/paper/2405.13629/) (2024)
- [Lifting Architectural Constraints of Injective Flows](https://mlsanity.com/paper/2306.01843/) (2023)
- [A Review of Change of Variable Formulas for Generative Modeling](https://mlsanity.com/paper/2308.02652/) (2023)
- [Semi-Autoregressive Energy Flows: Exploring Likelihood-Free Training of Normalizing Flows](https://mlsanity.com/paper/2206.06672/) (2022)
- [RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network](https://mlsanity.com/paper/2206.14098/) (2022)
- [Training Energy-Based Normalizing Flow with Score-Matching Objectives](https://mlsanity.com/paper/2305.15267/) (2023)
- [Orthonormal Convolutions for the Rotation Based Iterative Gaussianization](https://mlsanity.com/paper/2206.03860/) (2022)
- [FInC Flow: Fast and Invertible k × k Convolutions for Normalizing Flows](https://mlsanity.com/paper/2301.09266/) (2023)
- [Deep Generative Model based Rate-Distortion for Image Downscaling Assessment](https://mlsanity.com/paper/2403.15139/) (2024)
- [Hamiltonian Normalizing Flows as kinetic PDE solvers: application to the 1D Vlasov-Poisson Equations](https://mlsanity.com/paper/2505.04471/) (2025)
- [Parallel Backpropagation for Inverse of a Convolution with Application to Normalizing Flows](https://mlsanity.com/paper/2410.14634/) (2024)

## Abstract (arXiv)

Efficient gradient computation of the Jacobian determinant term is a core problem in many machine learning settings, and especially so in the normalizing flow framework. Most proposed flow models therefore either restrict to a function class with easy evaluation of the Jacobian determinant, or an efficient estimator thereof. However, these restrictions limit the performance of such density models, frequently requiring significant depth to reach desired performance levels. In this work, we propose Self Normalizing Flows, a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from O(D3) to O(D2), allowing for the training of flow architectures which were otherwise computationally infeasible, while also providing efficient sampling. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts, while training more quickly and surpassing the performance of functionally constrained counterparts.