# Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows

arXiv: https://arxiv.org/abs/2205.07918 · HTML: https://mlsanity.com/paper/2205.07918/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 27 of 27)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Variational Inference using Implicit Distributions](https://mlsanity.com/paper/1702.08235/) (2017)
- [Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization](https://mlsanity.com/paper/2006.10343/) (2020)
- [Automated Variational Inference in Probabilistic Programming](https://mlsanity.com/paper/1301.1299/) (2013)
- [Variational Inference with Tail-adaptive f-Divergence](https://mlsanity.com/paper/1810.11943/) (2018)
- [Variational auto-encoders with Student's t-prior](https://mlsanity.com/paper/2004.02581/) (2020)
- [Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder](https://mlsanity.com/paper/1906.00536/) (2019)
- [Variational Autoencoder with Embedded Student-t Mixture Model for Authorship Attribution](https://mlsanity.com/paper/2005.13930/) (2020)
- [Expectation Propagation for t-Exponential Family Using Q-Algebra](https://mlsanity.com/paper/1705.09046/) (2017)
- [Use of Student's t-Distribution for the Latent Layer in a Coupled Variational Autoencoder](https://mlsanity.com/paper/2011.10879/) (2020)

## Built on by (most cited first, 11 of 11)

- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Flexible Tails for Normalizing Flows](https://mlsanity.com/paper/2406.16971/) (2024)
- [Stable Training of Normalizing Flows for High-dimensional Variational Inference](https://mlsanity.com/paper/2402.16408/) (2024)
- [Multilevel neural simulation-based inference](https://mlsanity.com/paper/2506.06087/) (2025)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [A Heavy-Tailed Algebra for Probabilistic Programming](https://mlsanity.com/paper/2306.09262/) (2023)
- [Flexible Tails for Normalising Flows, with Application to the Modelling of Financial Return Data](https://mlsanity.com/paper/2311.00580/) (2023)
- [Fisher meets Feynman: score-based variational inference with a product of experts](https://mlsanity.com/paper/2510.21598/) (2025)
- [Preferential Normalizing Flows](https://mlsanity.com/paper/2410.08710/) (2024)
- [Stick-Breaking Mixture Normalizing Flows with Component-Wise Tail Adaptation for Variational Inference](https://mlsanity.com/paper/2510.07965/) (2025)
- [Radial Tail Repair: Post-Training Correction for Diffusion and Flow Models](https://mlsanity.com/paper/2610.16851/) (2026)

## Abstract (arXiv)

While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.