mlsanityData to 14 Oct 2026

Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing

arXiv 2112.04330 ↗First posted 8 Dec 2021stat.ML cs.IT cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

We consider the problem of signal estimation in generalized linear models defined via rotationally invariant design matrices. Since these matrices can have an arbitrary spectral distribution, this model is well suited for capturing complex correlation structures which often arise in applications. We propose a novel family of approximate message passing (AMP) algorithms for signal estimation, and rigorously characterize their performance in the high-dimensional limit via a state evolution recursion. Our rotationally invariant AMP has complexity of the same order as the existing AMP derived under the restrictive assumption of a Gaussian design; our algorithm also recovers this existing AMP as a special case. Numerical results showcase a performance close to Vector AMP (which is conjectured to be Bayes-optimal in some settings), but obtained with a much lower complexity, as the proposed algorithm does not require a computationally expensive singular value decomposition.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 23 of its 67 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

23 of 67 references.

References · 23

  1. 2014Phase Retrieval via Wirtinger Flow: Theory and Algorithmscited by 125
  2. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 116
  3. 2018A modern maximum-likelihood theory for high-dimensional logistic regressioncited by 81
  4. 2013Phase Retrieval using Alternating Minimizationcited by 58
  5. 2021A unifying tutorial on Approximate Message Passingcited by 50
  6. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  7. 2017Fundamental Limits of Weak Recovery with Applications to Phase Retrievalcited by 39
  8. 2020Phase retrieval in high dimensions: Statistical and computational phase transitionscited by 39
  9. 2020Approximate Message Passing algorithms for rotationally invariant matricescited by 32
  10. 2017Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimationcited by 26
  11. 2020Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)cited by 23
  12. 2014Phase transitions and sample complexity in Bayes-optimal matrix factorizationcited by 22
11 more
  1. 2020Approximate Message Passing with Spectral Initialization for Generalized Linear Modelscited by 22
  2. 2020The Numerics of Phase Retrievalcited by 21
  3. 2021Approximate Message Passing for orthogonally invariant ensembles: Multivariate non-linearities and spectral initializationcited by 18
  4. 2019Inference with Deep Generative Priors in High Dimensionscited by 17
  5. 2020Universality of Approximate Message Passing Algorithmscited by 16
  6. 2020All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimationcited by 16
  7. 2020Optimal Combination of Linear and Spectral Estimators for Generalized Linear Modelscited by 15
  8. 2021PCA Initialization for Approximate Message Passing in Rotationally Invariant Modelscited by 14
  9. 2019Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matricescited by 4
  10. 2021Memory Approximate Message Passingcited by 4
  11. 2016Self-Averaging Expectation Propagationcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 3; 2024: 8; 2025: 3.

18 papers.

Sort:

By area:

  1. 2022The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?cited by 9
  2. 2023Optimal Algorithms for the Inhomogeneous Spiked Wigner Modelcited by 8
  3. 2025Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recoverycited by 8
  4. 2025Optimal Spectral Transitions in High-Dimensional Multi-Index Modelscited by 7
  5. 2024Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noisecited by 6
  6. 2023Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passingcited by 5
  7. 2024Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depthcited by 5
  8. 2024Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methodscited by 5
  9. 2023QCM-SGM+: Improved Quantized Compressed Sensing With Score-Based Generative Modelscited by 4
  10. 2024A non-asymptotic distributional theory of approximate message passing for sparse and robust regressioncited by 4
  11. 2024Spectral Phase Transition and Optimal PCA in Block-Structured Spiked modelscited by 4
  12. 2024ROTI-GCV: Generalized Cross-Validation for right-ROTationally Invariant Datacited by 4
6 more
  1. 2024Unifying AMP Algorithms for Rotationally-Invariant Modelscited by 4
  2. 2022Bayes-optimal limits in structured PCA, and how to reach themcited by 3
  3. 2022Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Modelscited by 3
  4. 2021Sufficient-Statistic Memory AMPcited by 2
  5. 2024Stability of a Generalized Debiased Lasso with Applications to Resampling-Based Variable Selectioncited by 1
  6. 2025Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Modelscited by 1