# Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods

arXiv: https://arxiv.org/abs/2305.19079 · HTML: https://mlsanity.com/paper/2305.19079/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 53 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [Zero-Shot Noise2Noise: Efficient Image Denoising without any Data](https://mlsanity.com/paper/2303.11253/) (2023)
- [Zero-Shot Self-Supervised Learning for MRI Reconstruction](https://mlsanity.com/paper/2102.07737/) (2021)
- [Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation](https://mlsanity.com/paper/2005.03991/) (2020)
- [DSFormer: A Dual-domain Self-supervised Transformer for Accelerated Multi-contrast MRI Reconstruction](https://mlsanity.com/paper/2201.10776/) (2022)
- [HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction](https://mlsanity.com/paper/2203.08213/) (2022)
- [A theoretical framework for self-supervised MR image reconstruction using sub-sampling via variable density Noisier2Noise](https://mlsanity.com/paper/2205.10278/) (2022)
- [Noise2Fast: Fast Self-Supervised Single Image Blind Denoising](https://mlsanity.com/paper/2108.10209/) (2021)
- [Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework](https://mlsanity.com/paper/2109.12502/) (2021)

## Built on by (most cited first, 8 of 8)

- [Language models scale reliably with over-training and on downstream tasks](https://mlsanity.com/paper/2403.08540/) (2024)
- [Deep Learning for Accelerated and Robust MRI Reconstruction: a Review](https://mlsanity.com/paper/2404.15692/) (2024)
- [A Deep Learning Method for Simultaneous Denoising and Missing Wedge Reconstruction in Cryogenic Electron Tomography](https://mlsanity.com/paper/2311.05539/) (2023)
- [K-band: Self-supervised MRI Reconstruction via Stochastic Gradient Descent over K-space Subsets](https://mlsanity.com/paper/2308.02958/) (2023)
- [Self-Supervised Learning from Noisy and Incomplete Data](https://mlsanity.com/paper/2601.03244/) (2026)
- [Benchmarking Self-Supervised Learning Methods for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2502.14009/) (2025)
- [MotionTTT: 2D Test-Time-Training Motion Estimation for 3D Motion Corrected MRI](https://mlsanity.com/paper/2409.09370/) (2024)
- [Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction](https://mlsanity.com/paper/2501.03737/) (2025)

## Abstract (arXiv)

Supervised training of deep neural networks on pairs of clean image and noisy measurement achieves state-of-the-art performance for many image reconstruction tasks, but such training pairs are difficult to collect. Self-supervised methods enable training based on noisy measurements only, without clean images. In this work, we investigate the cost of self-supervised training in terms of sample complexity for a class of self-supervised methods that enable the computation of unbiased estimates of gradients of the supervised loss, including noise2noise methods. We analytically show that a model trained with such self-supervised training is as good as the same model trained in a supervised fashion, but self-supervised training requires more examples than supervised training. We then study self-supervised denoising and accelerated MRI empirically and characterize the cost of self-supervised training in terms of the number of additional samples required, and find that the performance gap between self-supervised and supervised training vanishes as a function of the training examples, at a problem-dependent rate, as predicted by our theory.