# OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage

arXiv: https://arxiv.org/abs/1306.6042 · HTML: https://mlsanity.com/paper/1306.6042/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2013.

## Built on by (most cited first, 14 of 14)

- [Nuclear Norm Regularization for Deep Learning](https://mlsanity.com/paper/2405.14544/) (2024)
- [Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCA](https://mlsanity.com/paper/2303.06198/) (2023)
- [Is your data alignable? Principled and interpretable alignability testing and integration of single-cell data](https://mlsanity.com/paper/2308.01839/) (2023)
- [Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods](https://mlsanity.com/paper/2405.13912/) (2024)
- [FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2506.09199/) (2025)
- [Capturing the Denoising Effect of PCA via Compression Ratio](https://mlsanity.com/paper/2204.10888/) (2022)
- [Generalization for Least Squares Regression With Simple Spiked Covariances](https://mlsanity.com/paper/2410.13991/) (2024)
- [eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization](https://mlsanity.com/paper/2605.02905/) (2026)
- [Design a Metric Robust to Complicated High Dimensional Noise for Efficient Manifold Denoising](https://mlsanity.com/paper/2401.03921/) (2024)
- [Random Matrix Theory-guided sparse PCA for single-cell RNA-seq data](https://mlsanity.com/paper/2509.15429/) (2025)
- [Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models](https://mlsanity.com/paper/2512.19334/) (2025)
- [On the Noise Sensitivity of the Randomized SVD](https://mlsanity.com/paper/2305.17435/) (2023)
- [Whitening Spherical Gaussian Mixtures in the Large-Dimensional Regime](https://mlsanity.com/paper/2509.17636/) (2025)
- [Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noise](https://mlsanity.com/paper/2602.03283/) (2026)

## Abstract (arXiv)

The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the_representation_ problem of how to best approximate a noisy measurement matrix using a low-rank matrix. Here, we consider the (unobservable)_denoising_ problem of how to best approximate a low-rank signal matrix buried in noise by optimal (re)weighting of the singular vectors of the measurement matrix. We exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the i.i.d. Gaussian noise case. Our analysis brings into sharp focus the shrinkage-and-thresholding form of the optimal weights, the non-convex nature of the associated shrinkage function (on the singular values) and explains why matrix regularization via singular value thresholding with convex penalty functions (such as the nuclear norm) will always be suboptimal. We validate our theoretical predictions with numerical simulations, develop an implementable algorithm (OptShrink) that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries.