# On the number of variables to use in principal component regression

arXiv: https://arxiv.org/abs/1906.01139 · HTML: https://mlsanity.com/paper/1906.01139/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 1 of 1)

- [Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties](https://mlsanity.com/paper/2205.07069/) (2022)

## Abstract (arXiv)

We study least squares linear regression over N uncorrelated Gaussian features that are selected in order of decreasing variance. When the number of selected features p is at most the sample size n, the estimator under consideration coincides with the principal component regression estimator; when p>n, the estimator is the least ℓ2 norm solution over the selected features. We give an average-case analysis of the out-of-sample prediction error as p,n,N → ∞ with p/N → α and n/N → β, for some constants α ∈ [0,1] and β ∈ (0,1). In this average-case setting, the prediction error exhibits a "double descent" shape as a function of p. We also establish conditions under which the minimum risk is achieved in the interpolating (p>n) regime.