# Self-Supervised Variational Auto-Encoders

arXiv: https://arxiv.org/abs/2010.02014 · HTML: https://mlsanity.com/paper/2010.02014/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Collapse-Proof Non-Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2410.04959/) (2024)
- [Variational Mixture of HyperGenerators for Learning Distributions Over Functions](https://mlsanity.com/paper/2302.06223/) (2023)
- [Unifying Self-Supervised Clustering and Energy-Based Models](https://mlsanity.com/paper/2401.00873/) (2023)
- [Super-resolution Reconstruction of Single Image for Latent features](https://mlsanity.com/paper/2211.12845/) (2022)
- [Quantum Down Sampling Filter for Variational Auto-encoder](https://mlsanity.com/paper/2501.06259/) (2025)

## Abstract (arXiv)

Density estimation, compression and data generation are crucial tasks in artificial intelligence. Variational Auto-Encoders (VAEs) constitute a single framework to achieve these goals. Here, we present a novel class of generative models, called self-supervised Variational Auto-Encoder (selfVAE), that utilizes deterministic and discrete variational posteriors. This class of models allows to perform both conditional and unconditional sampling, while simplifying the objective function. First, we use a single self-supervised transformation as a latent variable, where a transformation is either downscaling or edge detection. Next, we consider a hierarchical architecture, i.e., multiple transformations, and we show its benefits compared to the VAE. The flexibility of selfVAE in data reconstruction finds a particularly interesting use case in data compression tasks, where we can trade-off memory for better data quality, and vice-versa. We present performance of our approach on three benchmark image data (Cifar10, Imagenette64, and CelebA).