mlsanityData to 7 Oct 2026

Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods

arXiv 2305.19079 ↗First posted 30 May 2023eess.IV cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Supervised training of deep neural networks on pairs of clean image and noisy measurement achieves state-of-the-art performance for many image reconstruction tasks, but such training pairs are difficult to collect. Self-supervised methods enable training based on noisy measurements only, without clean images. In this work, we investigate the cost of self-supervised training in terms of sample complexity for a class of self-supervised methods that enable the computation of unbiased estimates of gradients of the supervised loss, including noise2noise methods. We analytically show that a model trained with such self-supervised training is as good as the same model trained in a supervised fashion, but self-supervised training requires more examples than supervised training. We then study self-supervised denoising and accelerated MRI empirically and characterize the cost of self-supervised training in terms of the number of additional samples required, and find that the performance gap between self-supervised and supervised training vanishes as a function of the training examples, at a problem-dependent rate, as predicted by our theory.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

37 of 53 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  7. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  8. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  9. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  10. 2017Deep Image Priorcited by 1,097
  11. 2021Scaling Vision Transformerscited by 993
  12. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
25 more
  1. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
  2. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 464
  3. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 340
  4. 2019Noise2Self: Blind Denoising by Self-Supervisioncited by 274
  5. 2018Unprocessing Images for Learned Raw Denoisingcited by 250
  6. 2017Benchmarking Denoising Algorithms with Real Photographscited by 209
  7. 2020End-to-End Variational Networks for Accelerated MRI Reconstructioncited by 201
  8. 2021Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Imagescited by 171
  9. 2019Noisier2Noise: Learning to Denoise from Unpaired Noisy Datacited by 125
  10. 2018Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networkscited by 120
  11. 2023Zero-Shot Noise2Noise: Efficient Image Denoising without any Datacited by 59
  12. 2021Zero-Shot Self-Supervised Learning for MRI Reconstructioncited by 53
  13. 2020Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximationcited by 39
  14. 2022DSFormer: A Dual-domain Self-supervised Transformer for Accelerated Multi-contrast MRI Reconstructioncited by 30
  15. 2022HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstructioncited by 30
  16. 2022A theoretical framework for self-supervised MR image reconstruction using sub-sampling via variable density Noisier2Noisecited by 29
  17. 2021Noise2Fast: Fast Self-Supervised Single Image Blind Denoisingcited by 28
  18. 2021Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Frameworkcited by 26
  19. 2021Data augmentation for deep learning based accelerated MRI reconstruction with limited datacited by 19
  20. 2021VORTEX: Physics-Driven Data Augmentations Using Consistency Training for Robust Accelerated MRI Reconstructioncited by 16
  21. 2020Neural Network-based Reconstruction in Compressed Sensing MRI Without Fully-sampled Training Datacited by 14
  22. 2022Scaling Laws For Deep Learning Based Image Reconstructioncited by 14
  23. 2018Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Datacited by 13
  24. 2023Evaluating Self-Supervised Learning via Risk Decompositioncited by 6
  25. 2023DAS-N2N: Machine learning Distributed Acoustic Sensing (DAS) signal denoising without clean datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 1.

8 papers.

  1. 2024Language models scale reliably with over-training and on downstream taskscited by 105
  2. 2024Deep Learning for Accelerated and Robust MRI Reconstruction: a Reviewcited by 9
  3. 2023A Deep Learning Method for Simultaneous Denoising and Missing Wedge Reconstruction in Cryogenic Electron Tomographycited by 7
  4. 2023K-band: Self-supervised MRI Reconstruction via Stochastic Gradient Descent over K-space Subsetscited by 5
  5. 2026Self-Supervised Learning from Noisy and Incomplete Datacited by 5
  6. 2025Benchmarking Self-Supervised Learning Methods for Accelerated MRI Reconstructioncited by 4
  7. 2024MotionTTT: 2D Test-Time-Training Motion Estimation for 3D Motion Corrected MRIcited by 2
  8. 2025Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstructioncited by 1