# t3-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergence

arXiv: https://arxiv.org/abs/2312.01133 · HTML: https://mlsanity.com/paper/2312.01133/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 2 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images](https://mlsanity.com/paper/2011.10650/) (2020)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Hierarchical VAEs Know What They Don't Know](https://mlsanity.com/paper/2102.08248/) (2021)
- [Stick-Breaking Variational Autoencoders](https://mlsanity.com/paper/1605.06197/) (2016)
- [Dirichlet Variational Autoencoder](https://mlsanity.com/paper/1901.02739/) (2019)
- [Nonparametric Variational Auto-encoders for Hierarchical Representation Learning](https://mlsanity.com/paper/1703.07027/) (2017)
- [Variational Inference based on Robust Divergences](https://mlsanity.com/paper/1710.06595/) (2017)
- [Marginal Tail-Adaptive Normalizing Flows](https://mlsanity.com/paper/2206.10311/) (2022)
- [Variational auto-encoders with Student's t-prior](https://mlsanity.com/paper/2004.02581/) (2020)
- [Robust model training and generalisation with Studentising flows](https://mlsanity.com/paper/2006.06599/) (2020)

## Built on by (most cited first, 18 of 18)

- [Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2410.14171/) (2024)
- [Go With the Flow: Fast Diffusion for Gaussian Mixture Models](https://mlsanity.com/paper/2412.09059/) (2024)
- [ProSAM: Enhancing the Robustness of SAM-based Visual Reference Segmentation with Probabilistic Prompts](https://mlsanity.com/paper/2506.21835/) (2025)
- [Multimodal Deep Generative Model for Semi-Supervised Learning under Class Imbalance](https://mlsanity.com/paper/2605.06289/) (2026)
- [ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation](https://mlsanity.com/paper/2608.10398/) (2026)
- [On Kernel-based Variational Autoencoder](https://mlsanity.com/paper/2405.12783/) (2024)
- [E4GEN: Event-level Explainable Extreme-Enhanced Time-series Generation](https://mlsanity.com/paper/2606.01634/) (2026)
- [Spherical Cauchy Variational Autoencoders: Heavy Angular Tails and Exact KL Evaluation](https://mlsanity.com/paper/2506.21278/) (2025)
- [Heavy-Tailed Class-Conditional Priors for Long-Tailed Generative Modeling](https://mlsanity.com/paper/2509.02154/) (2025)
- [Self-Regulating Annealing in Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2606.01645/) (2026)
- [Likelihood-Free Variational Autoencoders](https://mlsanity.com/paper/2504.17622/) (2025)
- [Metabolic cost of information processing in Poisson variational autoencoders](https://mlsanity.com/paper/2602.13421/) (2026)
- [Phase-Type Variational Autoencoders for Heavy-Tailed Data](https://mlsanity.com/paper/2603.01800/) (2026)
- [Surprisal-R'enyi Free Energy](https://mlsanity.com/paper/2603.03405/) (2026)
- [Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models](https://mlsanity.com/paper/2605.18931/) (2026)
- [An Extreme Value Perspective on Learning Stress Laws](https://mlsanity.com/paper/2607.10700/) (2026)
- [Generalized Graph Variational Autoencoders: Bounded Divergences Control Posterior Collapse](https://mlsanity.com/paper/2609.29546/) (2026)
- [Pro-t-VAE: Closed-Form Student's t Mixtures for Robust Prototype Discovery](https://mlsanity.com/paper/2610.14894/) (2026)

## Abstract (arXiv)

The variational autoencoder (VAE) typically employs a standard normal prior as a regularizer for the probabilistic latent encoder. However, the Gaussian tail often decays too quickly to effectively accommodate the encoded points, failing to preserve crucial structures hidden in the data. In this paper, we explore the use of heavy-tailed models to combat over-regularization. Drawing upon insights from information geometry, we propose t3VAE, a modified VAE framework that incorporates Student's t-distributions for the prior, encoder, and decoder. This results in a joint model distribution of a power form which we argue can better fit real-world datasets. We derive a new objective by reformulating the evidence lower bound as joint optimization of KL divergence between two statistical manifolds and replacing with γ-power divergence, a natural alternative for power families. t3VAE demonstrates superior generation of low-density regions when trained on heavy-tailed synthetic data. Furthermore, we show that t3VAE significantly outperforms other models on CelebA and imbalanced CIFAR-100 datasets.