mlsanityData to 7 Oct 2026

Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise

arXiv 2405.20993 ↗First posted 31 May 2024cs.IT cond-mat.dis-nn cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We consider a prototypical problem of Bayesian inference for a structured spiked model: a low-rank signal is corrupted by additive noise. While both information-theoretic and algorithmic limits are well understood when the noise is a Gaussian Wigner matrix, the more realistic case of structured noise still proves to be challenging. To capture the structure while maintaining mathematical tractability, a line of work has focused on rotationally invariant noise. However, existing studies either provide sub-optimal algorithms or are limited to special cases of noise ensembles. In this paper, using tools from statistical physics (replica method) and random matrix theory (generalized spherical integrals) we establish the first characterization of the information-theoretic limits for a noise matrix drawn from a general trace ensemble. Remarkably, our analysis unveils the asymptotic equivalence between the rotationally invariant model and a surrogate Gaussian one. Finally, we show how to saturate the predicted statistical limits using an efficient algorithm inspired by the theory of adaptive Thouless-Anderson-Palmer (TAP) equations.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 80 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

22 of 80 references.

References · 22

  1. 2017Community Detection and Stochastic Block Modelscited by 317
  2. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  3. 2018A modern maximum-likelihood theory for high-dimensional logistic regressioncited by 79
  4. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  5. 2020The estimation error of general first order methodscited by 33
  6. 2020Approximate Message Passing algorithms for rotationally invariant matricescited by 31
  7. 2016Mutual information for symmetric rank-one matrix estimation: A proof of the replica formulacited by 26
  8. 2020Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)cited by 23
  9. 2015MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channelcited by 22
  10. 2017The Computer Science and Physics of Community Detection: Landscapes, Phase Transitions, and Hardnesscited by 20
  11. 2016Message-passing algorithms for synchronization problems over compact groupscited by 18
  12. 2018Fundamental limits of detection in the spiked Wigner modelcited by 18
10 more
  1. 2021Approximate Message Passing for orthogonally invariant ensembles: Multivariate non-linearities and spectral initializationcited by 18
  2. 2021Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passingcited by 18
  3. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
  4. 2020Universality of Approximate Message Passing Algorithmscited by 16
  5. 2021PCA Initialization for Approximate Message Passing in Rotationally Invariant Modelscited by 14
  6. 2017Multi-Layer Generalized Linear Estimationcited by 12
  7. 2020Strong replica symmetry for high-dimensional disordered log-concave Gibbs measurescited by 6
  8. 2021Rank-one matrix estimation: analytic time evolution of gradient descent dynamicscited by 5
  9. 2022Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structurescited by 5
  10. 2024Optimality of Approximate Message Passing Algorithms for Spiked Matrix Models with Rotationally Invariant Noisecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2024On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariancecited by 4
  2. 2024Unifying AMP Algorithms for Rotationally-Invariant Modelscited by 3
  3. 2025Sparse corruption in low-rank matrix inference: the PCA benchmarkcited by 1
  4. 2025Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Modelscited by 1
  5. 2025Computational and statistical lower bounds for low-rank estimation under general inhomogeneous noisecited by 0
  6. 2026Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noisecited by 0