De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
Not on the map. Why?
Abstract (arXiv)
Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues. We study clean Gaussian random designs in which the aggregate covariance tail creates a nearly scalar sample-space floor comparable to the predictive head scale. De-floored principal component regression (dPCR) retains the cutoff and subtracts an estimated floor from the retained denominators. We prove an ordinary-PCR prediction-risk lower bound uniform over all ranks and a high-probability dPCR upper bound. When the floor is sharp and inexpensive to remove in population prediction risk, the conditional risk of dPCR is asymptotically negligible relative to that of the best ordinary PCR rank. An exact risk decomposition explains the separation: denominator inflation is governed by first spectral mass, whereas the clean prediction cost of correction is governed by squared spectral mass. A same-sample trimmed-mean floor estimate attains the oracle dPCR upper-bound rate at a prespecified rank, and the separation persists under approximate predictive alignment when the tail prediction-energy fraction vanishes. Separate pointwise fixed-aspect formulas show that the risk-optimal positive scalar correction improves rank-1 PCR, whereas mean-floor subtraction is generally not optimal for a broad Marchenko--Pastur bulk.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 18 Jul 2026.
Builds on
14 of 35 references.
References · 14
- 2019Benign Overfitting in Linear Regressioncited by 544
- 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
- 2020Benign overfitting in ridge regressioncited by 152
- 2019Harmless interpolation of noisy data in regressioncited by 121
- 2020On the Optimal Weighted ℓ2 Regularization in Overparameterized Linear Regressioncited by 97
- 2021Benign Overfitting of Constant-Stepsize SGD for Linear Regressioncited by 71
- 2022The Implicit Bias of Benign Overfittingcited by 38
- 2019On Robustness of Principal Component Regressioncited by 27
- 2021Foolish Crowds Support Benign Overfittingcited by 24
- 2020Dimensionality reduction, regularization, and generalization in overparameterized regressionscited by 13
- 2016Principal Component Projection Without Principal Component Analysiscited by 6
- 2023Adaptive Principal Component Regression with Applications to Panel Datacited by 4
2 more
Built on by
Not cited yet.