# Denoising Criterion for Variational Auto-Encoding Framework

arXiv: https://arxiv.org/abs/1511.06406 · HTML: https://mlsanity.com/paper/1511.06406/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2015.

## Built on by (most cited first, 30 of 43)

- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [Plug-and-Play split Gibbs sampler: embedding deep generative priors in Bayesian inference](https://mlsanity.com/paper/2304.11134/) (2023)
- [Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation](https://mlsanity.com/paper/2103.10693/) (2021)
- [My House, My Rules: Learning Tidying Preferences with Graph Neural Networks](https://mlsanity.com/paper/2111.03112/) (2021)
- [Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior](https://mlsanity.com/paper/2103.01648/) (2021)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Object-Centric Learning with Slot Mixture Module](https://mlsanity.com/paper/2311.04640/) (2023)
- [Vector-Quantized Vision Foundation Models for Object-Centric Learning](https://mlsanity.com/paper/2502.20263/) (2025)
- [VTAE: Variational Transformer Autoencoder with Manifolds Learning](https://mlsanity.com/paper/2304.00948/) (2023)
- [Length-Aware Motion Synthesis via Latent Diffusion](https://mlsanity.com/paper/2407.11532/) (2024)
- [Mask-GVAE: Blind Denoising Graphs via Partition](https://mlsanity.com/paper/2102.04228/) (2021)
- [Reasoning-Enhanced Object-Centric Learning for Videos](https://mlsanity.com/paper/2403.15245/) (2024)
- [Multi-Scale Fusion for Object Representation](https://mlsanity.com/paper/2410.01539/) (2024)
- [Exploring Versatile Prior for Human Motion via Motion Frequency Guidance](https://mlsanity.com/paper/2111.13074/) (2021)
- [Latent Space Bayesian Optimization with Latent Data Augmentation for Enhanced Exploration](https://mlsanity.com/paper/2302.02399/) (2023)
- [Implicit Variational Inference for High-Dimensional Posteriors](https://mlsanity.com/paper/2310.06643/) (2023)
- [Grouped Discrete Representation for Object-Centric Learning](https://mlsanity.com/paper/2411.02299/) (2024)
- [Latent Variable Models in the Era of Industrial Big Data: Extension and Beyond](https://mlsanity.com/paper/2208.10847/) (2022)
- [SQUWA: Signal Quality Aware DNN Architecture for Enhanced Accuracy in Atrial Fibrillation Detection from Noisy PPG Signals](https://mlsanity.com/paper/2404.15353/) (2024)
- [Blind Image Denoising and Inpainting Using Robust Hadamard Autoencoders](https://mlsanity.com/paper/2101.10876/) (2021)
- [PicTalky: Augmentative and Alternative Communication Software for Language Developmental Disabilities](https://mlsanity.com/paper/2109.12941/) (2021)
- [Hybrid BYOL-ViT: Efficient approach to deal with small datasets](https://mlsanity.com/paper/2111.04845/) (2021)
- [Training β-VAE by Aggregating a Learned Gaussian Posterior with a Decoupled Decoder](https://mlsanity.com/paper/2209.14783/) (2022)
- [Graph Harmony: Denoising and Nuclear-Norm Wasserstein Adaptation for Enhanced Domain Transfer in Graph-Structured Data](https://mlsanity.com/paper/2301.12361/) (2023)
- [Leveraging Image Augmentation for Object Manipulation: Towards Interpretable Controllability in Object-Centric Learning](https://mlsanity.com/paper/2310.08929/) (2023)
- [Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization](https://mlsanity.com/paper/2405.14221/) (2024)
- [SGW-based Multi-Task Learning in Vision Tasks](https://mlsanity.com/paper/2410.03778/) (2024)
- [AASAE: Augmentation-Augmented Stochastic Autoencoders](https://mlsanity.com/paper/2107.12329/) (2021)
- [A Bayesian Non-parametric Approach to Generative Models: Integrating Variational Autoencoder and Generative Adversarial Networks using Wasserstein and Maximum Mean Discrepancy](https://mlsanity.com/paper/2308.14048/) (2023)
- [inkn'hue: Enhancing Manga Colorization from Multiple Priors with Alignment Multi-Encoder VAE](https://mlsanity.com/paper/2311.01804/) (2023)

## Abstract (arXiv)

Denoising autoencoders (DAE) are trained to reconstruct their clean inputs with noise injected at the input level, while variational autoencoders (VAE) are trained with noise injected in their stochastic hidden layer, with a regularizer that encourages this noise injection. In this paper, we show that injecting noise both in input and in the stochastic hidden layer can be advantageous and we propose a modified variational lower bound as an improved objective function in this setup. When input is corrupted, then the standard VAE lower bound involves marginalizing the encoder conditional distribution over the input noise, which makes the training criterion intractable. Instead, we propose a modified training criterion which corresponds to a tractable bound when input is corrupted. Experimentally, we find that the proposed denoising variational autoencoder (DVAE) yields better average log-likelihood than the VAE and the importance weighted autoencoder on the MNIST and Frey Face datasets.