# Sparsity in Variational Autoencoders

arXiv: https://arxiv.org/abs/1812.07238 · HTML: https://mlsanity.com/paper/1812.07238/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2018.

## Built on by (most cited first, 6 of 6)

- [A survey on Variational Autoencoders from a GreenAI perspective](https://mlsanity.com/paper/2103.01071/) (2021)
- [Comparing the latent space of generative models](https://mlsanity.com/paper/2207.06812/) (2022)
- [A Sparsity-promoting Dictionary Model for Variational Autoencoders](https://mlsanity.com/paper/2203.15758/) (2022)
- [Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder](https://mlsanity.com/paper/2506.13658/) (2025)
- [Enhancing variational generation through self-decomposition](https://mlsanity.com/paper/2202.02738/) (2022)
- [A Multi-stage Constrained Optimization Framework for Data-driven Problems](https://mlsanity.com/paper/2607.23480/) (2026)

## Abstract (arXiv)

Working in high-dimensional latent spaces, the internal encoding of data in Variational Autoencoders becomes naturally sparse. We discuss this known but controversial phenomenon sometimes refereed to as overpruning, to emphasize the under-use of the model capacity. In fact, it is an important form of self-regularization, with all the typical benefits associated with sparsity: it forces the model to focus on the really important features, highly reducing the risk of overfitting. Especially, it is a major methodological guide for the correct tuning of the model capacity, progressively augmenting it to attain sparsity, or conversely reducing the dimension of the network removing links to zeroed out neurons. The degree of sparsity crucially depends on the network architecture: for instance, convolutional networks typically show less sparsity, likely due to the tighter relation of features to different spatial regions of the input.