# Batch-less stochastic gradient descent for compressive learning of deep regularization for image denoising

arXiv: https://arxiv.org/abs/2310.03085 · HTML: https://mlsanity.com/paper/2310.03085/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 12 of 12)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [Gradient Step Denoiser for convergent Plug-and-Play](https://mlsanity.com/paper/2110.03220/) (2021)
- [Adversarial Regularizers in Inverse Problems](https://mlsanity.com/paper/1805.11572/) (2018)
- [Learning local regularization for variational image restoration](https://mlsanity.com/paper/2102.06155/) (2021)
- [Sketching for Large-Scale Learning of Mixture Models](https://mlsanity.com/paper/1606.02838/) (2016)
- [Compressive Statistical Learning with Random Feature Moments](https://mlsanity.com/paper/1706.07180/) (2017)
- [Statistical Learning Guarantees for Compressive Clustering and Compressive Mixture Modeling](https://mlsanity.com/paper/2004.08085/) (2020)
- [Expressiveness of Rectifier Networks](https://mlsanity.com/paper/1511.05678/) (2015)
- [Sketching Datasets for Large-Scale Learning (long version)](https://mlsanity.com/paper/2008.01839/) (2020)
- [Compressive Classification (Machine Learning without learning)](https://mlsanity.com/paper/1812.01410/) (2018)
- [Compressive Learning of Generative Networks](https://mlsanity.com/paper/2002.05095/) (2020)

## Abstract (arXiv)

We consider the problem of denoising with the help of prior information taken from a database of clean signals or images. Denoising with variational methods is very efficient if a regularizer well adapted to the nature of the data is available. Thanks to the maximum a posteriori Bayesian framework, such regularizer can be systematically linked with the distribution of the data. With deep neural networks (DNN), complex distributions can be recovered from a large training database.To reduce the computational burden of this task, we adapt the compressive learning framework to the learning of regularizers parametrized by DNN. We propose two variants of stochastic gradient descent (SGD) for the recovery of deep regularization parameters from a heavily compressed database. These algorithms outperform the initially proposed method that was limited to low-dimensional signals, each iteration using information from the whole database. They also benefit from classical SGD convergence guarantees. Thanks to these improvements we show that this method can be applied for patch based image denoising.}