Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula

We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise Z corrupts a signal X, yielding the observation Y = X + σ Z with known σ ∈ (0,1). We propose universal denoisers, agnostic to both signal and noise distributions, that recover the signal distribution PX from PY. When the focus is on distributional recovery of PX rather than on individual realizations of X, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, which achieves O(σ2) accuracy. They shrink PY toward PX with O(σ4) and O(σ6) accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge--Amp`ere equation with higher-order accuracy and can be implemented efficiently via score matching. Let q denote the density of PY. For distributional denoising, we propose replacing the Bayes-optimal denoiser, T^*(y) = y + σ2 nabla log q(y), with denoisers exhibiting less-aggressive distributional shrinkage, T1(y) = y + fracσ22 nabla log q(y), T2(y) = y + fracσ22 nabla log q(y) - fracσ48 nabla !left( frac12 | nabla log q(y) |2 + nabla · nabla log q(y) right)!.