# Marginal Tail-Adaptive Normalizing Flows

arXiv: https://arxiv.org/abs/2206.10311 · HTML: https://mlsanity.com/paper/2206.10311/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 57 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [i-RevNet: Deep Invertible Networks](https://mlsanity.com/paper/1802.07088/) (2018)
- [Understanding and Mitigating Exploding Inverses in Invertible Neural Networks](https://mlsanity.com/paper/2006.09347/) (2020)
- [Transformation Autoregressive Networks](https://mlsanity.com/paper/1801.09819/) (2018)
- [High-Dimensional Probability Estimation with Deep Density Models](https://mlsanity.com/paper/1302.5125/) (2013)
- [Copula & Marginal Flows: Disentangling the Marginal from its Joint](https://mlsanity.com/paper/1907.03361/) (2019)
- [Robust model training and generalisation with Studentising flows](https://mlsanity.com/paper/2006.06599/) (2020)
- [COMET Flows: Towards Generative Modeling of Multivariate Extremes and Tail Dependence](https://mlsanity.com/paper/2205.01224/) (2022)
- [Scale-free Networks Well Done](https://mlsanity.com/paper/1811.02071/) (2018)
- [Copula-based synthetic data augmentation for machine-learning emulators](https://mlsanity.com/paper/2012.09037/) (2020)
- [Copula-Based Normalizing Flows](https://mlsanity.com/paper/2107.07352/) (2021)

## Built on by (most cited first, 12 of 12)

- [Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2410.14171/) (2024)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics](https://mlsanity.com/paper/2309.01156/) (2023)
- [t3-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergence](https://mlsanity.com/paper/2312.01133/) (2023)
- [Flexible Tails for Normalizing Flows](https://mlsanity.com/paper/2406.16971/) (2024)
- [A VAE Approach to Sample Multivariate Extremes](https://mlsanity.com/paper/2306.10987/) (2023)
- [Flexible Tails for Normalising Flows, with Application to the Modelling of Financial Return Data](https://mlsanity.com/paper/2311.00580/) (2023)
- [Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators](https://mlsanity.com/paper/2308.13498/) (2023)
- [Stick-Breaking Mixture Normalizing Flows with Component-Wise Tail Adaptation for Variational Inference](https://mlsanity.com/paper/2510.07965/) (2025)
- [Reducing Estimation Uncertainty Using Normalizing Flows and Stratification](https://mlsanity.com/paper/2602.10706/) (2026)
- [Correcting Source Mismatch in Flow Matching with Radial-Angular Transport](https://mlsanity.com/paper/2604.04291/) (2026)
- [Tail Annealing for Heavy-Tailed Flow Matching](https://mlsanity.com/paper/2605.20068/) (2026)

## Abstract (arXiv)

Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy -- especially on the tails of the distribution -- and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.