mlsanityData to 7 Oct 2026

The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior

arXiv 2304.11409 ↗First posted 22 Apr 2023eess.IV cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep Image Prior (DIP) shows that some network architectures naturally bias towards smooth images and resist noises, a phenomenon known as spectral bias. Image denoising is an immediate application of this property. Although DIP has removed the requirement of large training sets, it still presents two practical challenges for denoising: architectural design and noise-fitting, which are often intertwined. Existing methods mostly handcraft or search for the architecture from a large design space, due to the lack of understanding on how the architectural choice corresponds to the image. In this study, we analyze from a frequency perspective to demonstrate that the unlearnt upsampling is the main driving force behind the denoising phenomenon in DIP. This finding then leads to strategies for estimating a suitable architecture for every image without a laborious search. Extensive experiments show that the estimated architectures denoise and preserve the textural details better than current methods with up to 95% fewer parameters. The under-parameterized nature also makes them especially robust to a higher level of noise.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 22 Apr 2023.

Builds on

28 of 40 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  4. 2018On the Spectral Bias of Neural Networkscited by 1,321
  5. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  7. 2017Deep Image Priorcited by 1,097
  8. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  9. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 407
  10. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 278
  11. 2016Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applicationscited by 244
  12. 2018Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networkscited by 120
16 more
  1. 2018"Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priorscited by 114
  2. 2019DeepRED: Deep Image Prior Powered by REDcited by 84
  3. 2018Real-world Noisy Image Denoising: A New Benchmarkcited by 74
  4. 2021A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detectioncited by 68
  5. 2018Image Restoration using Total Variation Regularized Deep Image Priorcited by 65
  6. 2021On the Frequency Bias of Generative Modelscited by 60
  7. 2019A Bayesian Perspective on the Deep Image Priorcited by 52
  8. 2021Early Stopping for Deep Image Priorcited by 44
  9. 2018Training Deep Learning Based Denoisers without Ground Truth Datacited by 43
  10. 2019Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generatorscited by 40
  11. 2021Rethinking Deep Image Prior for Denoisingcited by 38
  12. 2021On Measuring and Controlling the Spectral Bias of the Deep Image Priorcited by 36
  13. 2019The Spectral Bias of the Deep Image Priorcited by 20
  14. 2020NAS-DIP: Learning Deep Image Prior with Neural Architecture Searchcited by 20
  15. 2020Neural Architecture Search for Deep Image Priorcited by 14
  16. 2021ISNAS-DIP: Image-Specific Neural Architecture Search for Deep Image Priorcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 2; 2026: 2.

11 papers.

  1. 2024Playing to Vision Foundation Model's Strengths in Stereo Matchingcited by 23
  2. 2024GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Searchcited by 13
  3. 2024LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusioncited by 12
  4. 2025YOND: Practical Blind Raw Image Denoising Free from Camera-Specific Data Dependencycited by 4
  5. 2023Towards Architecture-Agnostic Untrained Network Priors for Image Reconstruction with Frequency Regularizationcited by 3
  6. 2024Masked Pre-training Enables Universal Zero-shot Denoisercited by 3
  7. 2024Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initializationcited by 2
  8. 2024Robustness and Exploration of Variational and Machine Learning Approaches to Inverse Problems: An Overviewcited by 1
  9. 2025Zero-Shot CFC: Fast Real-World Image Denoising based on Cross-Frequency Consistencycited by 0
  10. 2026Automating Parameter Selection in Deep Image Prior for Fluorescence Microscopy Image Denoising via Similarity-Based Parameter Transfercited by 0
  11. 2026Pooling-Based Context Modeling for Convolution-Free Deep Image Priorcited by 0