# Stein's unbiased risk estimate and Hyv"arinen's score matching

arXiv: https://arxiv.org/abs/2502.20123 · HTML: https://mlsanity.com/paper/2502.20123/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2025.

## Built on by (most cited first, 8 of 8)

- [Proper scoring rules for estimation and forecast evaluation](https://mlsanity.com/paper/2504.01781/) (2025)
- [Universal priors: solving empirical Bayes via Bayesian inference and pretraining](https://mlsanity.com/paper/2602.15136/) (2026)
- [Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula](https://mlsanity.com/paper/2511.09500/) (2025)
- [Distributional Shrinkage II: Higher-Order Scores Encode Brenier Map](https://mlsanity.com/paper/2512.09295/) (2025)
- [Empirical Bayes for correlated Gaussian sequence model](https://mlsanity.com/paper/2607.03596/) (2026)
- [Differentially Private Nonparametric Modal Learning with Applications to Regression and Clustering](https://mlsanity.com/paper/2607.29675/) (2026)
- [DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing](https://mlsanity.com/paper/2608.02769/) (2026)
- [Posterior contraction rates in Sobolev norms and Bayesian derivative estimation for infinite-dimensional exponential families](https://mlsanity.com/paper/2608.11130/) (2026)

## Abstract (arXiv)

Given a collection of observed signals corrupted with Gaussian noise, how can we learn to optimally denoise them? This fundamental problem arises in both empirical Bayes and generative modeling. In empirical Bayes, the predominant approach is via nonparametric maximum likelihood estimation (NPMLE), while in generative modeling, score matching (SM) methods have proven very successful. In our setting, Hyv"arinen's implicit SM is equivalent to another classical idea from statistics---Stein's Unbiased Risk Estimate (SURE). Revisiting SURE minimization, we establish, for the first time, that SURE achieves nearly parametric rates of convergence of the regret in the classical empirical Bayes setting with homoscedastic noise. We also prove that SURE-training can achieve fast rates of convergence to the oracle denoiser in a commonly studied misspecified model. We show how to practically implement our method in settings involving heteroscedasticity and side-information, such as in an application to the estimation of economic mobility in the Opportunity Atlas. Our empirical results demonstrate the superior performance of SURE-training over NPMLE under misspecification. Collectively, our findings advance SURE/SM as a strong alternative to the NPMLE for empirical Bayes problems in both theory and practice.