# Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula

arXiv: https://arxiv.org/abs/2511.09500 · HTML: https://mlsanity.com/paper/2511.09500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [The probability flow ODE is provably fast](https://mlsanity.com/paper/2305.11798/) (2023)
- [Generative Modeling with Denoising Auto-Encoders and Langevin Sampling](https://mlsanity.com/paper/2002.00107/) (2020)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Statistical Efficiency of Score Matching: The View from Isoperimetry](https://mlsanity.com/paper/2210.00726/) (2022)
- [Neural Empirical Bayes](https://mlsanity.com/paper/1903.02334/) (2019)
- [Density Estimation in Infinite Dimensional Exponential Families](https://mlsanity.com/paper/1312.3516/) (2013)
- [Deep Energy Estimator Networks](https://mlsanity.com/paper/1805.08306/) (2018)
- [Estimating Certain Integral Probability Metric (IPM) is as Hard as Estimating under the IPM](https://mlsanity.com/paper/1911.00730/) (2019)
- [Reversible Gromov-Monge Sampler for Simulation-Based Inference](https://mlsanity.com/paper/2109.14090/) (2021)
- [Optimal convex M-estimation via score matching](https://mlsanity.com/paper/2403.16688/) (2024)
- [Stein's unbiased risk estimate and Hyv"arinen's score matching](https://mlsanity.com/paper/2502.20123/) (2025)
- [Denoising Diffusions with Optimal Transport: Localization, Curvature, and Multi-Scale Complexity](https://mlsanity.com/paper/2411.01629/) (2024)
- [No-Regret Generative Modeling via Parabolic Monge-Amp`ere PDE](https://mlsanity.com/paper/2504.09279/) (2025)
- [Constrained Denoising, Empirical Bayes, and Optimal Transport](https://mlsanity.com/paper/2506.09986/) (2025)
- [A New Perspective On Denoising Based On Optimal Transport](https://mlsanity.com/paper/2312.08135/) (2023)

## Abstract (arXiv)

We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise Z corrupts a signal X, yielding the observation Y = X + σ Z with known σ ∈ (0,1). We propose universal denoisers, agnostic to both signal and noise distributions, that recover the signal distribution PX from PY. When the focus is on distributional recovery of PX rather than on individual realizations of X, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, which achieves O(σ2) accuracy. They shrink PY toward PX with O(σ4) and O(σ6) accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge--Amp`ere equation with higher-order accuracy and can be implemented efficiently via score matching. Let q denote the density of PY. For distributional denoising, we propose replacing the Bayes-optimal denoiser, T^*(y) = y + σ2 nabla log q(y), with denoisers exhibiting less-aggressive distributional shrinkage, T1(y) = y + fracσ22 nabla log q(y), T2(y) = y + fracσ22 nabla log q(y) - fracσ48 nabla !left( frac12 | nabla log q(y) |2 + nabla · nabla log q(y) right)!.