# Inference with Deep Generative Priors in High Dimensions

arXiv: https://arxiv.org/abs/1911.03409 · HTML: https://mlsanity.com/paper/1911.03409/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 17 of 17)

- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Learned reconstruction methods with convergence guarantees](https://mlsanity.com/paper/2206.05431/) (2022)
- [Instance-Optimal Compressed Sensing via Posterior Sampling](https://mlsanity.com/paper/2106.11438/) (2021)
- [Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing](https://mlsanity.com/paper/2112.04330/) (2021)
- [Label-Free Explainability for Unsupervised Models](https://mlsanity.com/paper/2203.01928/) (2022)
- [The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?](https://mlsanity.com/paper/2205.10009/) (2022)
- [Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions](https://mlsanity.com/paper/2201.08082/) (2022)
- [Inverse Problems Leveraging Pre-trained Contrastive Representations](https://mlsanity.com/paper/2110.07439/) (2021)
- [Unifying AMP Algorithms for Rotationally-Invariant Models](https://mlsanity.com/paper/2412.01574/) (2024)
- [Asymptotics of Ridge Regression in Convolutional Models](https://mlsanity.com/paper/2103.04557/) (2021)
- [Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models](https://mlsanity.com/paper/2212.01572/) (2022)
- [Sufficient-Statistic Memory AMP](https://mlsanity.com/paper/2112.15327/) (2021)
- [Fast and Robust Phase Retrieval via Deep Expectation-Consistent Approximation](https://mlsanity.com/paper/2407.09687/) (2024)
- [Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions](https://mlsanity.com/paper/2505.18046/) (2025)
- [Fundamental limits and algorithms for sparse linear regression with sublinear sparsity](https://mlsanity.com/paper/2101.11156/) (2021)
- [Bayesian imaging inverse problem with SA-Roundtrip prior via HMC-pCN sampler](https://mlsanity.com/paper/2310.17817/) (2023)
- [Inferring Change Points in High-Dimensional Regression via Approximate Message Passing](https://mlsanity.com/paper/2404.07864/) (2024)

## Abstract (arXiv)

Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful in practice, rigorous performance analysis is complicated by the non-convex nature of the underlying optimization problems. This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks. ML-VAMP can be configured to compute maximum a priori (MAP) or approximate minimum mean-squared error (MMSE) estimates for these networks. We show that the performance of ML-VAMP can be exactly predicted in a certain high-dimensional random limit. Furthermore, under certain conditions, ML-VAMP yields estimates that achieve the minimum (i.e., Bayes-optimal) MSE as predicted by the replica method. In this way, ML-VAMP provides a computationally efficient method for multi-layer inference with an exact performance characterization and testable conditions for optimality in the large-system limit.