The Risk Shadow of Principal Component Analysis: When 99.9999% Variance Preservation Causes Catastrophic Decision Errors
Not on the map. Why?
Abstract (arXiv)
Principal Component Analysis (PCA) preserves variance, not the information needed to detect rare catastrophic events. This paper proves the existence of a it Risk Shadow: PCA can retain over 99.9999 percent of total variance while completely erasing all signal about rare, high-impact failures. When this happens, even the best possible classifier operating on the PCA representation reduces to a constant predictor. The root cause is a fundamental mismatch between variance maximization and tail risk awareness. To break the shadow, we introduce Expectile PCA (ExPCA) and Tail-Preserving PCA (TP-PCA), two methods that reweight the data covariance toward high-impact events. We prove theoretically that ExPCA strictly outperforms PCA in retaining rare-event information, and we validate our claims on synthetic data and a real-world credit card fraud detection benchmark. Our results call for a fundamental rethinking of variance-based dimensionality reduction in high-stakes decisions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 6 of its 69 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
6 of 69 references.
References · 6
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
- 2018The Price of Fair PCA: One Extra Dimensioncited by 67
- 2018Dataset: Rare Event Classification in Multivariate Time Seriescited by 7
- 2025Breaking the Barriers of Text-Hungry and Audio-Deficient AIcited by 1
- 2026Causal Mirage Equilibrium in Agentic Machine Intelligencecited by 1
Built on by
Not cited yet.