# Conditional Variational Autoencoder for Learned Image Reconstruction

arXiv: https://arxiv.org/abs/2110.11681 · HTML: https://mlsanity.com/paper/2110.11681/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 60 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Solving ill-posed inverse problems using iterative deep neural networks](https://mlsanity.com/paper/1704.04058/) (2017)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Bayesian Deep Ensembles via the Neural Tangent Kernel](https://mlsanity.com/paper/2007.05864/) (2020)
- [An Uncertain Future: Forecasting from Static Images using Variational Autoencoders](https://mlsanity.com/paper/1606.07873/) (2016)
- [A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction](https://mlsanity.com/paper/1610.09736/) (2016)
- [Recurrent Inference Machines for Solving Inverse Problems](https://mlsanity.com/paper/1706.04008/) (2017)
- [Controlling Neural Networks via Energy Dissipation](https://mlsanity.com/paper/1904.03081/) (2019)
- [Quantifying Model Uncertainty in Inverse Problems via Bayesian Deep Gradient Descent](https://mlsanity.com/paper/2007.09971/) (2020)
- [Quantifying Sources of Uncertainty in Deep Learning-Based Image Reconstruction](https://mlsanity.com/paper/2011.08413/) (2020)
- [Variational Gaussian Approximation for Poisson Data](https://mlsanity.com/paper/1709.05885/) (2017)
- [Probabilistic Residual Learning for Aleatoric Uncertainty in Image Restoration](https://mlsanity.com/paper/1908.01010/) (2019)

## Built on by (most cited first, 13 of 13)

- [Equivariant Bootstrapping for Uncertainty Quantification in Imaging Inverse Problems](https://mlsanity.com/paper/2310.11838/) (2023)
- [Uncertainty Estimation and Propagation in Accelerated MRI Reconstruction](https://mlsanity.com/paper/2308.02631/) (2023)
- [Compressed Sensing MRI Reconstruction Regularized by VAEs with Structured Image Covariance](https://mlsanity.com/paper/2210.14586/) (2022)
- [Graph Structure Learning with Interpretable Bayesian Neural Networks](https://mlsanity.com/paper/2406.14786/) (2024)
- [Can Generative Models Improve Self-Supervised Representation Learning?](https://mlsanity.com/paper/2403.05966/) (2024)
- [VAE-iForest: Auto-encoding Reconstruction and Isolation-based Anomalies Detecting Fallen Objects on Road Surface](https://mlsanity.com/paper/2203.01193/) (2022)
- [High-dimensional reliability-based design optimization using stochastic emulators](https://mlsanity.com/paper/2604.05759/) (2026)
- [Towards Open-World Co-Salient Object Detection with Generative Uncertainty-aware Group Selective Exchange-Masking](https://mlsanity.com/paper/2310.10264/) (2023)
- [Drivetrain simulation using variational autoencoders](https://mlsanity.com/paper/2501.17653/) (2025)
- [Hazy Pedestrian Trajectory Prediction via Physical Priors and Graph-Mamba](https://mlsanity.com/paper/2509.24020/) (2025)
- [Heterogeneity in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2512.22941/) (2025)
- [Variational Sparse Paired Autoencoders (vsPAIR) for Inverse Problems and Uncertainty Quantification](https://mlsanity.com/paper/2602.02948/) (2026)
- [Deep unfolding of MCMC kernels: scalable, modular & explainable GANs for high-dimensional posterior sampling](https://mlsanity.com/paper/2602.20758/) (2026)

## Abstract (arXiv)

Learned image reconstruction techniques using deep neural networks have recently gained popularity, and have delivered promising empirical results. However, most approaches focus on one single recovery for each observation, and thus neglect the uncertainty information. In this work, we develop a novel computational framework that approximates the posterior distribution of the unknown image at each query observation. The proposed framework is very flexible: It handles implicit noise models and priors, it incorporates the data formation process (i.e., the forward operator), and the learned reconstructive properties are transferable between different datasets. Once the network is trained using the conditional variational autoencoder loss, it provides a computationally efficient sampler for the approximate posterior distribution via feed-forward propagation, and the summarizing statistics of the generated samples are used for both point-estimation and uncertainty quantification. We illustrate the proposed framework with extensive numerical experiments on positron emission tomography (with both moderate and low count levels) showing that the framework generates high-quality samples when compared with state-of-the-art methods.