# Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

arXiv: https://arxiv.org/abs/2008.03326 · HTML: https://mlsanity.com/paper/2008.03326/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2020.

## Built on by (most cited first, 15 of 15)

- [A unifying tutorial on Approximate Message Passing](https://mlsanity.com/paper/2105.02180/) (2021)
- [Hessian Eigenspectra of More Realistic Nonlinear Models](https://mlsanity.com/paper/2103.01519/) (2021)
- [Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem](https://mlsanity.com/paper/2103.04902/) (2021)
- [Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing](https://mlsanity.com/paper/2112.04330/) (2021)
- [Generalization Guarantees for Neural Architecture Search with Train-Validation Split](https://mlsanity.com/paper/2104.14132/) (2021)
- [PCA Initialization for Approximate Message Passing in Rotationally Invariant Models](https://mlsanity.com/paper/2106.02356/) (2021)
- [Optimal Algorithms for the Inhomogeneous Spiked Wigner Model](https://mlsanity.com/paper/2302.06665/) (2023)
- [Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery](https://mlsanity.com/paper/2502.01583/) (2025)
- [Optimal Spectral Transitions in High-Dimensional Multi-Index Models](https://mlsanity.com/paper/2502.02545/) (2025)
- [Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing](https://mlsanity.com/paper/2308.14507/) (2023)
- [Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods](https://mlsanity.com/paper/2405.13912/) (2024)
- [Spectral Phase Transition and Optimal PCA in Block-Structured Spiked models](https://mlsanity.com/paper/2403.03695/) (2024)
- [Fundamental limits of Non-Linear Low-Rank Matrix Estimation](https://mlsanity.com/paper/2403.04234/) (2024)
- [Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap](https://mlsanity.com/paper/2205.13527/) (2022)
- [Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models](https://mlsanity.com/paper/2512.19334/) (2025)

## Abstract (arXiv)

We study the problem of recovering an unknown signal boldsymbol x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator hatboldsymbol xrm L and a spectral estimator hatboldsymbol xrm s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine hatboldsymbol xrm L and hatboldsymbol xrm s. At the heart of our analysis is the exact characterization of the joint empirical distribution of (boldsymbol x, hatboldsymbol xrm L, hatboldsymbol xrm s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of hatboldsymbol xrm L and hatboldsymbol xrm s, given the limiting distribution of the signal boldsymbol x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θhatboldsymbol xrm L+hatboldsymbol xrm s and we derive the optimal combination coefficient. In order to establish the limiting distribution of (boldsymbol x, hatboldsymbol xrm L, hatboldsymbol xrm s), we design and analyze an Approximate Message Passing (AMP) algorithm whose iterates give hatboldsymbol xrm L and approach hatboldsymbol xrm s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.