# Capturing the Denoising Effect of PCA via Compression Ratio

arXiv: https://arxiv.org/abs/2204.10888 · HTML: https://mlsanity.com/paper/2204.10888/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 35 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 5 of 5)

- [ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions](https://mlsanity.com/paper/2201.00382/) (2022)
- [Optimality of Spectral Clustering in the Gaussian Mixture Model](https://mlsanity.com/paper/1911.00538/) (2019)
- [A simple SVD algorithm for finding hidden partitions](https://mlsanity.com/paper/1404.3918/) (2014)
- [OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage](https://mlsanity.com/paper/1306.6042/) (2013)
- [Finite Sample Guarantees for PCA in Non-Isotropic and Data-Dependent Noise](https://mlsanity.com/paper/1709.06255/) (2017)

## Built on by (most cited first, 3 of 3)

- [PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Models](https://mlsanity.com/paper/2511.06315/) (2025)
- [Confident Clustering via PCA Compression Ratio and Its Application to Single-cell RNA-seq Analysis](https://mlsanity.com/paper/2205.09849/) (2022)
- [On the Power of SVD in the Stochastic Block Model](https://mlsanity.com/paper/2309.15322/) (2023)

## Abstract (arXiv)

Principal component analysis (PCA) is one of the most fundamental tools in machine learning with broad use as a dimensionality reduction and denoising tool. In the later setting, while PCA is known to be effective at subspace recovery and is proven to aid clustering algorithms in some specific settings, its improvement of noisy data is still not well quantified in general. In this paper, we propose a novel metric called compression ratio to capture the effect of PCA on high-dimensional noisy data. We show that, for data with underlying community structure, PCA significantly reduces the distance of data points belonging to the same community while reducing inter-community distance relatively mildly. We explain this phenomenon through both theoretical proofs and experiments on real-world data. Building on this new metric, we design a straightforward algorithm that could be used to detect outliers. Roughly speaking, we argue that points that have a lower variance of compression ratio do not share a common signal with others (hence could be considered outliers). We provide theoretical justification for this simple outlier detection algorithm and use simulations to demonstrate that our method is competitive with popular outlier detection tools. Finally, we run experiments on real-world high-dimension noisy data (single-cell RNA-seq) to show that removing points from these datasets via our outlier detection method improves the accuracy of clustering algorithms. Our method is very competitive with popular outlier detection tools in this task.