mlsanityData to 7 Oct 2026

On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance

arXiv 2411.01974 ↗First posted 4 Nov 2024cond-mat.dis-nn cs.IT cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Matrix denoising is central to signal processing and machine learning. Its statistical analysis when the matrix to infer has a factorised structure with a rank growing proportionally to its dimension remains a challenge, except when it is rotationally invariant. In this case the information theoretic limits and an efficient Bayes-optimal denoising algorithm, called rotational invariant estimator [1,2], are known. Beyond this setting few results can be found. The reason is that the model is not a usual spin system because of the growing rank dimension, nor a matrix model (as appearing in high-energy physics) due to the lack of rotation symmetry, but rather a hybrid between the two. Here we make progress towards the understanding of Bayesian matrix denoising when the signal is a factored matrix XX^intercal that is not rotationally invariant. Monte Carlo simulations suggest the existence of a denoising-factorisation transition separating a phase where denoising using the rotational invariant estimator remains Bayes-optimal due to universality properties of the same nature as in random matrix theory, from one where universality breaks down and better denoising is possible, though algorithmically hard. We argue that it is only beyond the transition that factorisation, i.e., estimating X itself, becomes possible up to irresolvable ambiguities. On the theory side, we combine mean-field techniques in an interpretable multiscale fashion in order to access the minimum mean-square error and mutual information. Interestingly, our alternative method yields equations reproducible by the replica approach of [3]. Using numerical insights, we delimit the portion of phase diagram where we conjecture the mean-field theory to be exact, and correct it using universality when it is not. Our complete ansatz matches well the numerics in the whole phase diagram when considering finite size effects.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 157 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

27 of 157 references.

References · 27

  1. 2017Community Detection and Stochastic Block Modelscited by 317
  2. 2022High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representationcited by 146
  3. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  4. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  5. 2019Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratiocited by 82
  6. 2023How Two-Layer Neural Networks Learn, One (Giant) Step at a Timecited by 67
  7. 2020Reducibility and Statistical-Computational Gaps from Secret Leakagecited by 50
  8. 2018Reducibility and Computational Lower Bounds for Problems with Planted Sparse Structurecited by 35
  9. 2018Optimality and Sub-optimality of PCA I: Spiked Random Matrix Modelscited by 33
  10. 2016Mutual information for symmetric rank-one matrix estimation: A proof of the replica formulacited by 26
  11. 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 25
  12. 2024Bayes-optimal learning of an extensive-width neural network from quadratically many samplescited by 23
15 more
  1. 2014Phase transitions and sample complexity in Bayes-optimal matrix factorizationcited by 22
  2. 2015MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channelcited by 22
  3. 2018Rank-one matrix estimation: analysis of algorithmic and information theoretic limits by the spatial coupling methodcited by 11
  4. 2021Statistical limits of dictionary learning: random matrix theory and the spectral replica methodcited by 10
  5. 2018Glassy nature of the hard phase in inference problemscited by 9
  6. 2020Information-Theoretic Limits for the Matrix Tensor Productcited by 9
  7. 2021Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networkscited by 9
  8. 2022The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?cited by 9
  9. 2013Phase Diagram and Approximate Message Passing for Blind Calibration and Dictionary Learningcited by 6
  10. 2022Matrix factorization with neural networkscited by 6
  11. 2024Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noisecited by 6
  12. 2024Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methodscited by 5
  13. 2013Sample Complexity of Bayesian Optimal Dictionary Learningcited by 2
  14. 2023The Decimation Scheme for Symmetric Matrix Factorizationcited by 2
  15. 2023Gradient flow on extensive-rank positive semi-definite matrix denoisingcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Statistical mechanics of extensive-width Bayesian neural networks near interpolationcited by 9
  2. 2025Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regimecited by 6
  3. 2025Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolationcited by 0
  4. 2025Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolationcited by 0