# Compressed Sensing MRI Reconstruction Regularized by VAEs with Structured Image Covariance

arXiv: https://arxiv.org/abs/2210.14586 · HTML: https://mlsanity.com/paper/2210.14586/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 25 of 25)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty](https://mlsanity.com/paper/2006.06015/) (2020)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](https://mlsanity.com/paper/1709.00753/) (2017)
- [Plug and play methods for magnetic resonance imaging (long version)](https://mlsanity.com/paper/1903.08616/) (2019)
- [Structured Uncertainty Prediction Networks](https://mlsanity.com/paper/1802.07079/) (2018)
- [Modeling Sparse Deviations for Compressed Sensing using Generative Models](https://mlsanity.com/paper/1807.01442/) (2018)
- [Regularising Inverse Problems with Generative Machine Learning Models](https://mlsanity.com/paper/2107.11191/) (2021)
- [A Generative Variational Model for Inverse Problems in Imaging](https://mlsanity.com/paper/2104.12630/) (2021)
- [Conditional Variational Autoencoder for Learned Image Reconstruction](https://mlsanity.com/paper/2110.11681/) (2021)
- [Correction by Projection: Denoising Images with Generative Adversarial Networks](https://mlsanity.com/paper/1803.04477/) (2018)
- [Training VAEs Under Structured Residuals](https://mlsanity.com/paper/1804.01050/) (2018)
- [Robust Image Reconstruction with Misaligned Structural Information](https://mlsanity.com/paper/2004.00589/) (2020)

## Built on by (most cited first, 4 of 4)

- [Variational Bayes image restoration with compressive autoencoders](https://mlsanity.com/paper/2311.17744/) (2023)
- [Deep priors for satellite image restoration with accurate uncertainties](https://mlsanity.com/paper/2412.04130/) (2024)
- [Multiparameter Uncertainty Mapping in Quantitative Molecular MRI using a Physics-Structured Variational Autoencoder (PS-VAE)](https://mlsanity.com/paper/2602.03317/) (2026)
- [It depends: Incorporating correlations for joint aleatoric and epistemic uncertainties of high-dimensional output spaces](https://mlsanity.com/paper/2608.24518/) (2026)

## Abstract (arXiv)

Objective: This paper investigates how generative models, trained on ground-truth images, can be used changesas priors for inverse problems, penalizing reconstructions far from images the generator can produce. The aim is that learned regularization will provide complex data-driven priors to inverse problems while still retaining the control and insight of a variational regularization method. Moreover, unsupervised learning, without paired training data, allows the learned regularizer to remain flexible to changes in the forward problem such as noise level, sampling pattern or coil sensitivities in MRI. Approach: We utilize variational autoencoders (VAEs) that generate not only an image but also a covariance uncertainty matrix for each image. The covariance can model changing uncertainty dependencies caused by structure in the image, such as edges or objects, and provides a new distance metric from the manifold of learned images. Main results: We evaluate these novel generative regularizers on retrospectively sub-sampled real-valued MRI measurements from the fastMRI dataset. We compare our proposed learned regularization against other unlearned regularization approaches and unsupervised and supervised deep learning methods. Significance: Our results show that the proposed method is competitive with other state-of-the-art methods and behaves consistently with changing sampling patterns and noise levels.