# Tackling Over-pruning in Variational Autoencoders

arXiv: https://arxiv.org/abs/1706.03643 · HTML: https://mlsanity.com/paper/1706.03643/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2017.

## Built on by (most cited first, 14 of 14)

- [Posterior Collapse and Latent Variable Non-identifiability](https://mlsanity.com/paper/2301.00537/) (2023)
- [A survey on Variational Autoencoders from a GreenAI perspective](https://mlsanity.com/paper/2103.01071/) (2021)
- [REX: Revisiting Budgeted Training with an Improved Schedule](https://mlsanity.com/paper/2107.04197/) (2021)
- [Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Network](https://mlsanity.com/paper/2304.12770/) (2023)
- [Improved Training of Sparse Coding Variational Autoencoder via Weight Normalization](https://mlsanity.com/paper/2101.09453/) (2021)
- [Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder](https://mlsanity.com/paper/2506.13658/) (2025)
- [Neural Variational Dropout Processes](https://mlsanity.com/paper/2510.19425/) (2025)
- [Enhancing variational generation through self-decomposition](https://mlsanity.com/paper/2202.02738/) (2022)
- [Sparsity-Inducing Categorical Prior Improves Robustness of the Information Bottleneck](https://mlsanity.com/paper/2203.02592/) (2022)
- [Genetic Constrained Graph Variational Autoencoder for COVID-19 Drug Discovery](https://mlsanity.com/paper/2104.11674/) (2021)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [A Study of Representational Properties of Unsupervised Anomaly Detection in Brain MRI](https://mlsanity.com/paper/2211.15527/) (2022)
- [A Multi-stage Constrained Optimization Framework for Data-driven Problems](https://mlsanity.com/paper/2607.23480/) (2026)
- [Soft-Constrained Optimization of Latent Space in Variational Autoencoders](https://mlsanity.com/paper/2607.23751/) (2026)

## Abstract (arXiv)

Variational autoencoders (VAE) are directed generative models that learn factorial latent variables. As noted by Burda et al. (2015), these models exhibit the problem of factor over-pruning where a significant number of stochastic factors fail to learn anything and become inactive. This can limit their modeling power and their ability to learn diverse and meaningful latent representations. In this paper, we evaluate several methods to address this problem and propose a more effective model-based approach called the epitomic variational autoencoder (eVAE). The so-called epitomes of this model are groups of mutually exclusive latent factors that compete to explain the data. This approach helps prevent inactive units since each group is pressured to explain the data. We compare the approaches with qualitative and quantitative results on MNIST and TFD datasets. Our results show that eVAE makes efficient use of model capacity and generalizes better than VAE.