# Precise Performance of Linear Denoisers in the Proportional Regime

arXiv: https://arxiv.org/abs/2603.18483 · HTML: https://mlsanity.com/paper/2603.18483/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [A Model of Double Descent for High-dimensional Binary Linear Classification](https://mlsanity.com/paper/1911.05822/) (2019)
- [Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions](https://mlsanity.com/paper/2106.03791/) (2021)
- [The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression](https://mlsanity.com/paper/2201.05149/) (2022)
- [Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation](https://mlsanity.com/paper/2302.08923/) (2023)
- [Regularized Linear Regression for Binary Classification](https://mlsanity.com/paper/2311.02270/) (2023)
- [Universality in Transfer Learning for Linear Models](https://mlsanity.com/paper/2410.02164/) (2024)
- [A Novel Gaussian Min-Max Theorem and its Applications](https://mlsanity.com/paper/2402.07356/) (2024)
- [Universality of High-Dimensional Logistic Regression and a Novel CGMT under Dependence with Applications to Data Augmentation](https://mlsanity.com/paper/2502.15752/) (2025)

## Abstract (arXiv)

In the present paper we study the performance of linear denoisers for noisy data of the form x + z, where x ∈ ℝd is the desired data with zero mean and unknown covariance Sigma, and z ~ N(0, Sigmaz) is additive noise. Since the covariance Sigma is not known, the standard Wiener filter cannot be employed for denoising. Instead we assume we are given samples x1,dots,xn ∈ ℝd from the true distribution. A standard approach would then be to estimate Sigma from the samples and use it to construct an ``empirical" Wiener filter. However, in this paper, motivated by the denoising step in diffusion models, we take a different approach whereby we train a linear denoiser W from the data itself. In particular, we synthetically construct noisy samples hatxi of the data by injecting the samples with Gaussian noise with covariance Sigma1 neq Sigmaz and find the best W that approximates Whatxi ≈ xi in a least-squares sense. In the proportional regime fracnd rightarrow kappa > 1 we use the it Convex Gaussian Min-Max Theorem (CGMT) to analytically find the closed form expression for the generalization error of the denoiser obtained from this process. Using this expression one can optimize over Sigma1 to find the best possible denoiser. Our numerical simulations show that our denoiser outperforms the ``empirical" Wiener filter in many scenarios and approaches the optimal Wiener filter as kapparightarrow∞.