# Rate-Optimal Denoising with Deep Neural Networks

arXiv: https://arxiv.org/abs/1805.08855 · HTML: https://mlsanity.com/paper/1805.08855/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 5 of 5)

- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Discovering Structure From Corruption for Unsupervised Image Reconstruction](https://mlsanity.com/paper/2304.05589/) (2023)
- [The Star Geometry of Critic-Based Regularizer Learning](https://mlsanity.com/paper/2408.16852/) (2024)
- [Deep ReLU networks -- injectivity capacity upper bounds](https://mlsanity.com/paper/2412.19677/) (2024)
- [Signal Recovery with Non-Expansive Generative Network Priors](https://mlsanity.com/paper/2204.13599/) (2022)

## Abstract (arXiv)

Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy observation. The underlying principle is that neural networks trained on large datasets have empirically been shown to be able to generate natural images well from a low-dimensional latent representation of the image. Given such a generator network, a noisy image can be denoised by i) finding the closest image in the range of the generator or by ii) passing it through an encoder-generator architecture (known as an autoencoder). However, there is little theory to justify this success, let alone to predict the denoising performance as a function of the network parameters. In this paper we consider the problem of denoising an image from additive Gaussian noise using the two generator based approaches. In both cases, we assume the image is well described by a deep neural network with ReLU activations functions, mapping a k-dimensional code to an n-dimensional image. In the case of the autoencoder, we show that the feedforward network reduces noise energy by a factor of O(k/n). In the case of optimizing over the range of a generative model, we state and analyze a simple gradient algorithm that minimizes a non-convex loss function, and provably reduces noise energy by a factor of O(k/n). We also demonstrate in numerical experiments that this denoising performance is, indeed, achieved by generative priors learned from data.