mlsanityData to 7 Oct 2026

De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction

arXiv 2607.16638 ↗First posted 18 Jul 2026math.ST cs.LG stat.MEcited by 0

Not on the map. Why?

Abstract (arXiv)

Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues. We study clean Gaussian random designs in which the aggregate covariance tail creates a nearly scalar sample-space floor comparable to the predictive head scale. De-floored principal component regression (dPCR) retains the cutoff and subtracts an estimated floor from the retained denominators. We prove an ordinary-PCR prediction-risk lower bound uniform over all ranks and a high-probability dPCR upper bound. When the floor is sharp and inexpensive to remove in population prediction risk, the conditional risk of dPCR is asymptotically negligible relative to that of the best ordinary PCR rank. An exact risk decomposition explains the separation: denominator inflation is governed by first spectral mass, whereas the clean prediction cost of correction is governed by squared spectral mass. A same-sample trimmed-mean floor estimate attains the oracle dPCR upper-bound rate at a prespecified rank, and the separation persists under approximate predictive alignment when the tail prediction-energy fraction vanishes. Separate pointwise fixed-aspect formulas show that the risk-optimal positive scalar correction improves rank-1 PCR, whereas mean-floor subtraction is generally not optimal for a broad Marchenko--Pastur bulk.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 18 Jul 2026.

Builds on

14 of 35 references.

References · 14

  1. 2019Benign Overfitting in Linear Regressioncited by 544
  2. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  3. 2020Benign overfitting in ridge regressioncited by 152
  4. 2019Harmless interpolation of noisy data in regressioncited by 121
  5. 2020On the Optimal Weighted ℓ2 Regularization in Overparameterized Linear Regressioncited by 97
  6. 2021Benign Overfitting of Constant-Stepsize SGD for Linear Regressioncited by 71
  7. 2022The Implicit Bias of Benign Overfittingcited by 38
  8. 2019On Robustness of Principal Component Regressioncited by 27
  9. 2021Foolish Crowds Support Benign Overfittingcited by 24
  10. 2020Dimensionality reduction, regularization, and generalization in overparameterized regressionscited by 13
  11. 2016Principal Component Projection Without Principal Component Analysiscited by 6
  12. 2023Adaptive Principal Component Regression with Applications to Panel Datacited by 4
2 more
  1. 2016Faster Principal Component Regression and Stable Matrix Chebyshev Approximationcited by 3
  2. 2018Sketching for Principal Component Regressioncited by 2

Built on by

Not cited yet.