# Autoencoding Variational Autoencoder

arXiv: https://arxiv.org/abs/2012.03715 · HTML: https://mlsanity.com/paper/2012.03715/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 25 of 25)

- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [HSIGene: A Foundation Model For Hyperspectral Image Generation](https://mlsanity.com/paper/2409.12470/) (2024)
- [Scale-MIA: A Scalable Model Inversion Attack against Secure Federated Learning via Latent Space Reconstruction](https://mlsanity.com/paper/2311.05808/) (2023)
- [DiffuseTrace: A Transparent and Flexible Watermarking Scheme for Latent Diffusion Model](https://mlsanity.com/paper/2405.02696/) (2024)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Certifiably Robust Variational Autoencoders](https://mlsanity.com/paper/2102.07559/) (2021)
- [Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data](https://mlsanity.com/paper/2501.13483/) (2025)
- [Exploring the Latent Space of Autoencoders with Interventional Assays](https://mlsanity.com/paper/2106.16091/) (2021)
- [Alleviating Adversarial Attacks on Variational Autoencoders with MCMC](https://mlsanity.com/paper/2203.09940/) (2022)
- [Generalization Gap in Amortized Inference](https://mlsanity.com/paper/2205.11640/) (2022)
- [RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editing](https://mlsanity.com/paper/2603.19206/) (2026)
- [Lifelong Generative Modelling Using Dynamic Expansion Graph Model](https://mlsanity.com/paper/2112.08370/) (2021)
- [Bayesian Transfer Learning](https://mlsanity.com/paper/2312.13484/) (2023)
- [Leverage Variational Graph Representation For Model Poisoning on Federated Learning](https://mlsanity.com/paper/2404.15042/) (2024)
- [Semantic-Enhanced Time-Series Forecasting via Large Language Models](https://mlsanity.com/paper/2508.07697/) (2025)
- [Graph Representation-based Model Poisoning on the Heterogeneous Internet of Agents](https://mlsanity.com/paper/2511.07176/) (2025)
- [Learned Image Transmission with Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2408.16340/) (2024)
- [Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs](https://mlsanity.com/paper/2605.07961/) (2026)
- [Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels](https://mlsanity.com/paper/2605.18846/) (2026)
- [Robust VAEs via Generating Process of Noise Augmented Data](https://mlsanity.com/paper/2407.18632/) (2024)
- [PH-VAE: A Polynomial Hierarchical Variational Autoencoder Towards Disentangled Representation Learning](https://mlsanity.com/paper/2502.02856/) (2025)
- [Generative Bayesian Filtering for State Estimation](https://mlsanity.com/paper/2607.20521/) (2026)
- [A Multi-stage Constrained Optimization Framework for Data-driven Problems](https://mlsanity.com/paper/2607.23480/) (2026)
- [Soft-Constrained Optimization of Latent Space in Variational Autoencoders](https://mlsanity.com/paper/2607.23751/) (2026)
- [Perfect Reconstruction Cannot Show Whether Two Codebooks on a VAE's Latent Space Agree](https://mlsanity.com/paper/2610.14012/) (2026)

## Abstract (arXiv)

Does a Variational AutoEncoder (VAE) consistently encode typical samples generated from its decoder? This paper shows that the perhaps surprising answer to this question is `No'; a (nominally trained) VAE does not necessarily amortize inference for typical samples that it is capable of generating. We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. Our approach hinges on an alternative construction of the variational approximation distribution to the true posterior of an extended VAE model with a Markov chain alternating between the encoder and the decoder. The method can be used to train a VAE model from scratch or given an already trained VAE, it can be run as a post processing step in an entirely self supervised way without access to the original training data. Our experimental analysis reveals that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks. We provide experimental results on the ColorMnist and CelebA benchmark datasets that quantify the properties of the learned representations and compare the approach with a baseline that is specifically trained for the desired property.