mlsanityData to 7 Oct 2026

A unifying tutorial on Approximate Message Passing

arXiv 2105.02180 ↗First posted 5 May 2021math.ST cs.IT math.ITcited by 49

Not on the map. Why?

Abstract (arXiv)

Over the last decade or so, Approximate Message Passing (AMP) algorithms have become extremely popular in various structured high-dimensional statistical problems. The fact that the origins of these techniques can be traced back to notions of belief propagation in the statistical physics literature lends a certain mystique to the area for many statisticians. Our goal in this work is to present the main ideas of AMP from a statistical perspective, to illustrate the power and flexibility of the AMP framework. Along the way, we strengthen and unify many of the results in the existing literature.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 17 of its 153 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

17 of 153 references.

References · 17

  1. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 146
  2. 2018The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regressioncited by 45
  3. 2020A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-ℓ1-Norm Interpolated Classifierscited by 44
  4. 2018Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regressioncited by 42
  5. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  6. 2020Approximate Message Passing algorithms for rotationally invariant matricescited by 31
  7. 2014Phase transitions and sample complexity in Bayes-optimal matrix factorizationcited by 22
  8. 2016Finite Sample Analysis of Approximate Message Passing Algorithmscited by 20
  9. 2019High-dimensional principal component analysis with heterogeneous missingnesscited by 16
  10. 2020All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimationcited by 16
  11. 2020Sharp Concentration Results for Heavy-Tailed Distributionscited by 15
  12. 2020Optimal Combination of Linear and Spectral Estimators for Generalized Linear Modelscited by 15
5 more
  1. 2019Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passingcited by 14
  2. 2015Capacity-achieving Sparse Superposition Codes via Approximate Message Passing Decodingcited by 5
  3. 2019Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matricescited by 4
  4. 2019Asymptotics of MAP Inference in Deep Networkscited by 4
  5. 2014Statistical Estimation: From Denoising to Sparse Regression and Hidden Cliquescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 5; 2023: 12; 2024: 10; 2025: 11; 2026: 4.

49 papers.

Sort:

By area:

  1. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39
  2. 2022Theoretical Perspectives on Deep Learning Methods in Inverse Problemscited by 22
  3. 2021Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passingcited by 18
  4. 2025Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regimecited by 18
  5. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
  6. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  7. 2021PCA Initialization for Approximate Message Passing in Rotationally Invariant Modelscited by 14
  8. 2025Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formulacited by 11
  9. 2025The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networkscited by 11
  10. 2021Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspectivecited by 10
  11. 2022The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?cited by 9
  12. 2023Approximate message passing from random initialization with applications to mathbbZ2 synchronizationcited by 9
37 more
  1. 2023Optimal Algorithms for the Inhomogeneous Spiked Wigner Modelcited by 8
  2. 2023Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCAcited by 7
  3. 2023Causal Message Passing for Experiments with Unknown and General Network Interferencecited by 7
  4. 2023Approximate Message Passing for the Matrix Tensor Product Modelcited by 6
  5. 2023Semidefinite programs simulate approximate message passing robustlycited by 6
  6. 2026Optimal scaling laws in learning hierarchical multi-index modelscited by 6
  7. 2021Minimum ℓ1-norm interpolators: Precise asymptotics and multiple descentcited by 5
  8. 2022A New Central Limit Theorem for the Augmented IPW Estimator: Variance Inflation, Cross-Fit Covariance and Beyondcited by 5
  9. 2023Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passingcited by 5
  10. 2024Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methodscited by 5
  11. 2025Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Modelcited by 5
  12. 2024Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depthcited by 4
  13. 2024A replica analysis of under-baggingcited by 4
  14. 2024Optimal thresholds and algorithms for a model of multi-modal learning in high dimensionscited by 4
  15. 2022Bayes-optimal limits in structured PCA, and how to reach themcited by 3
  16. 2022Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Modelscited by 3
  17. 2023Mixed Regression via Approximate Message Passingcited by 3
  18. 2023Multinomial Logistic Regression: Asymptotic Normality on Null Covariates in High-Dimensionscited by 3
  19. 2024Fast, robust approximate message passingcited by 3
  20. 2024Unifying AMP Algorithms for Rotationally-Invariant Modelscited by 3
  21. 2021Sufficient-Statistic Memory AMPcited by 2
  22. 2023Bayes optimal learning in high-dimensional linear regression with network side informationcited by 2
  23. 2024Fundamental limits of Non-Linear Low-Rank Matrix Estimationcited by 2
  24. 2025Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scalingcited by 2
  25. 2025Dimension-Free Bounds for Generalized First-Order Methods via Gaussian Couplingcited by 2
  26. 2025Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributionscited by 2
  27. 2025Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theorycited by 1
  28. 2025Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Modelscited by 1
  29. 2021Towards Designing Optimal Sensing Matrices for Generalized Linear Inverse Problemscited by 0
  30. 2023Estimation of embedding vectors in high dimensionscited by 0
  31. 2024Inferring Change Points in High-Dimensional Regression via Approximate Message Passingcited by 0
  32. 2024Robust Random Graph Matching in Dense Graphs via an Approximate Message Passing Type Algorithmcited by 0
  33. 2025Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passingcited by 0
  34. 2025Sharp Trade-Offs in High-Dimensional Inference via 2-Level SLOPEcited by 0
  35. 2026Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passingcited by 0
  36. 2026Easy, robust approximate message passing for planted spike modelscited by 0
  37. 2026DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passingcited by 0