# CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse

arXiv: https://arxiv.org/abs/2309.02968 · HTML: https://mlsanity.com/paper/2309.02968/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Local Aggregation for Unsupervised Learning of Visual Embeddings](https://mlsanity.com/paper/1903.12355/) (2019)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [Robust and Generalizable Visual Representation Learning via Random Convolutions](https://mlsanity.com/paper/2007.13003/) (2020)
- [Lagging Inference Networks and Posterior Collapse in Variational Autoencoders](https://mlsanity.com/paper/1901.05534/) (2019)
- [Semi-Amortized Variational Autoencoders](https://mlsanity.com/paper/1802.02550/) (2018)
- [Informative Dropout for Robust Representation Learning: A Shape-bias Perspective](https://mlsanity.com/paper/2008.04254/) (2020)
- [Avoiding Latent Variable Collapse With Generative Skip Models](https://mlsanity.com/paper/1807.04863/) (2018)
- [Return-Based Contrastive Representation Learning for Reinforcement Learning](https://mlsanity.com/paper/2102.10960/) (2021)
- [On the Generalization of Representations in Reinforcement Learning](https://mlsanity.com/paper/2203.00543/) (2022)
- [Learning Representations by Maximizing Mutual Information in Variational Autoencoders](https://mlsanity.com/paper/1912.13361/) (2019)
- [A Tutorial on VAEs: From Bayes' Rule to Lossless Compression](https://mlsanity.com/paper/2006.10273/) (2020)
- [Forget-me-not! Contrastive Critics for Mitigating Posterior Collapse](https://mlsanity.com/paper/2207.09535/) (2022)

## Built on by (most cited first, 4 of 4)

- [PcLast: Discovering Plannable Continuous Latent States](https://mlsanity.com/paper/2311.03534/) (2023)
- [A Comprehensive Survey on Network Traffic Synthesis: From Statistical Models to Deep Learning](https://mlsanity.com/paper/2507.01976/) (2025)
- [ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders](https://mlsanity.com/paper/2407.06797/) (2024)
- [Toward Architecture-Agnostic Local Control of Posterior Collapse in VAEs](https://mlsanity.com/paper/2508.12530/) (2025)

## Abstract (arXiv)

The Variational Autoencoder (VAE) is known to suffer from the phenomenon of textitposterior collapse, where the latent representations generated by the model become independent of the inputs. This leads to degenerated representations of the input, which is attributed to the limitations of the VAE's objective function. In this work, we propose a novel solution to this issue, the Contrastive Regularization for Variational Autoencoders (CR-VAE). The core of our approach is to augment the original VAE with a contrastive objective that maximizes the mutual information between the representations of similar visual inputs. This strategy ensures that the information flow between the input and its latent representation is maximized, effectively avoiding posterior collapse. We evaluate our method on a series of visual datasets and demonstrate, that CR-VAE outperforms state-of-the-art approaches in preventing posterior collapse.