# On Robustness of Principal Component Regression

arXiv: https://arxiv.org/abs/1902.10920 · HTML: https://mlsanity.com/paper/1902.10920/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2019.

## Built on by (most cited first, 28 of 28)

- [Causal Matrix Completion](https://mlsanity.com/paper/2109.15154/) (2021)
- [Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventions](https://mlsanity.com/paper/2303.14226/) (2023)
- [CausalSim: A Causal Framework for Unbiased Trace-Driven Simulation](https://mlsanity.com/paper/2201.01811/) (2022)
- [Survival Kernets: Scalable and Interpretable Deep Kernel Survival Analysis with an Accuracy Guarantee](https://mlsanity.com/paper/2206.10477/) (2022)
- [Near Optimal Private and Robust Linear Regression](https://mlsanity.com/paper/2301.13273/) (2023)
- [Synthetically Controlled Bandits](https://mlsanity.com/paper/2202.07079/) (2022)
- [Random features models: a way to study the success of naive imputation](https://mlsanity.com/paper/2402.03839/) (2024)
- [Federated Optimization of Smooth Loss Functions](https://mlsanity.com/paper/2201.01954/) (2022)
- [Learning Treatment Effects in Panels with General Intervention Patterns](https://mlsanity.com/paper/2106.02780/) (2021)
- [Errors-in-variables Fr'echet Regression with Low-rank Covariate Approximation](https://mlsanity.com/paper/2305.09282/) (2023)
- [Adaptive Principal Component Regression with Applications to Panel Data](https://mlsanity.com/paper/2307.01357/) (2023)
- [Tangential Wasserstein Projections](https://mlsanity.com/paper/2207.14727/) (2022)
- [Strategyproof Decision-Making in Panel Data Settings and Beyond](https://mlsanity.com/paper/2211.14236/) (2022)
- [Doubly Robust Inference in Causal Latent Factor Models](https://mlsanity.com/paper/2402.11652/) (2024)
- [ROTI-GCV: Generalized Cross-Validation for right-ROTationally Invariant Data](https://mlsanity.com/paper/2406.11666/) (2024)
- [Differentially Private Synthetic Control](https://mlsanity.com/paper/2303.14084/) (2023)
- [Can Linear Probes Measure LLM Uncertainty?](https://mlsanity.com/paper/2510.04108/) (2025)
- [Network Synthetic Interventions: A Causal Framework for Panel Data Under Network Interference](https://mlsanity.com/paper/2210.11355/) (2022)
- [A Causal Inference Framework for Data Rich Environments](https://mlsanity.com/paper/2504.01702/) (2025)
- [Online Linear Regression with Paid Stochastic Features](https://mlsanity.com/paper/2511.08073/) (2025)
- [De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction](https://mlsanity.com/paper/2607.16638/) (2026)
- [Fast exact recovery of noisy matrix from few entries: the infinity norm approach](https://mlsanity.com/paper/2501.19224/) (2025)
- [Calibrated Principal Component Regression](https://mlsanity.com/paper/2510.19020/) (2025)
- [Handling Covariate Mismatch in Collaborative Linear Prediction](https://mlsanity.com/paper/2602.02083/) (2026)
- [Improved Guarantees for Heterogeneous Treatment-Effect Estimation via Matrix Completion](https://mlsanity.com/paper/2605.30319/) (2026)
- [On the Oracle Complexity of Interpolation-Based Gradient Descent](https://mlsanity.com/paper/2606.19878/) (2026)
- [Synthetic Nearest Neighbors: Extending Synthetic Controls for Matrix Completion with Missing Not at Random Data](https://mlsanity.com/paper/2609.13586/) (2026)
- [Beyond Second-Order Information: Paid Logistic Regression with Noisy Features](https://mlsanity.com/paper/2610.12919/) (2026)

## Abstract (arXiv)

Principal component regression (PCR) is a simple, but powerful and ubiquitously utilized method. Its effectiveness is well established when the covariates exhibit low-rank structure. However, its ability to handle settings with noisy, missing, and mixed-valued, i.e., discrete and continuous, covariates is not understood and remains an important open challenge. As the main contribution of this work we establish the robustness of PCR, without any change, in this respect and provide meaningful finite-sample analysis. To do so, we establish that PCR is equivalent to performing linear regression after pre-processing the covariate matrix via hard singular value thresholding (HSVT). As a result, in the context of counterfactual analysis using observational data, we show PCR is equivalent to the recently proposed robust variant of the synthetic control method, known as robust synthetic control (RSC). As an immediate consequence, we obtain finite-sample analysis of the RSC estimator that was previously absent. As an important contribution to the synthetic controls literature, we establish that an (approximate) linear synthetic control exists in the setting of a generalized factor model or latent variable model; traditionally in the literature, the existence of a synthetic control needs to be assumed to exist as an axiom. We further discuss a surprising implication of the robustness property of PCR with respect to noise, i.e., PCR can learn a good predictive model even if the covariates are tactfully transformed to preserve differential privacy. Finally, this work advances the state-of-the-art analysis for HSVT by establishing stronger guarantees with respect to the ℓ2, ∞-norm rather than the Frobenius norm as is commonly done in the matrix estimation literature, which may be of interest in its own right.