# ISNAS-DIP: Image-Specific Neural Architecture Search for Deep Image Prior

arXiv: https://arxiv.org/abs/2111.15362 · HTML: https://mlsanity.com/paper/2111.15362/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 27 Nov 2021.

## Builds on (most cited first, 19 of 19)

- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [DeepRED: Deep Image Prior Powered by RED](https://mlsanity.com/paper/1903.10176/) (2019)
- [Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators](https://mlsanity.com/paper/1910.14634/) (2019)
- [On Measuring and Controlling the Spectral Bias of the Deep Image Prior](https://mlsanity.com/paper/2107.01125/) (2021)
- [NAS-DIP: Learning Deep Image Prior with Neural Architecture Search](https://mlsanity.com/paper/2008.11713/) (2020)
- [Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search](https://mlsanity.com/paper/1803.00370/) (2018)
- [Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising](https://mlsanity.com/paper/1909.08228/) (2019)

## Built on by (most cited first, 12 of 12)

- [AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration](https://mlsanity.com/paper/2309.10438/) (2023)
- [GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Search](https://mlsanity.com/paper/2405.20725/) (2024)
- [The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior](https://mlsanity.com/paper/2304.11409/) (2023)
- [Priors in Deep Image Restoration and Enhancement: A Survey](https://mlsanity.com/paper/2206.02070/) (2022)
- [Towards Architecture-Agnostic Untrained Network Priors for Image Reconstruction with Frequency Regularization](https://mlsanity.com/paper/2312.09988/) (2023)
- [Masked Pre-training Enables Universal Zero-shot Denoiser](https://mlsanity.com/paper/2401.14966/) (2024)
- [Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization](https://mlsanity.com/paper/2406.05288/) (2024)
- [Chasing Better Deep Image Priors between Over- and Under-parameterization](https://mlsanity.com/paper/2410.24187/) (2024)
- [DIPLI: Deep Image Prior Lucky Imaging for Blind Astronomical Image Restoration](https://mlsanity.com/paper/2503.15984/) (2025)
- [Net2Net: When Un-trained Meets Pre-trained Networks for Robust Real-World Denoising](https://mlsanity.com/paper/2510.02733/) (2025)
- [Automating Parameter Selection in Deep Image Prior for Fluorescence Microscopy Image Denoising via Similarity-Based Parameter Transfer](https://mlsanity.com/paper/2601.12055/) (2026)
- [Pooling-Based Context Modeling for Convolution-Free Deep Image Prior](https://mlsanity.com/paper/2607.02952/) (2026)

## Abstract (arXiv)

Recent works show that convolutional neural network (CNN) architectures have a spectral bias towards lower frequencies, which has been leveraged for various image restoration tasks in the Deep Image Prior (DIP) framework. The benefit of the inductive bias the network imposes in the DIP framework depends on the architecture. Therefore, researchers have studied how to automate the search to determine the best-performing model. However, common neural architecture search (NAS) techniques are resource and time-intensive. Moreover, best-performing models are determined for a whole dataset of images instead of for each image independently, which would be prohibitively expensive. In this work, we first show that optimal neural architectures in the DIP framework are image-dependent. Leveraging this insight, we then propose an image-specific NAS strategy for the DIP framework that requires substantially less training than typical NAS approaches, effectively enabling image-specific NAS. We justify the proposed strategy's effectiveness by (1) demonstrating its performance on a NAS Dataset for DIP that includes 522 models from a particular search space (2) conducting extensive experiments on image denoising, inpainting, and super-resolution tasks. Our experiments show that image-specific metrics can reduce the search space to a small cohort of models, of which the best model outperforms current NAS approaches for image restoration. Codes and datasets are available at https://github.com/ozgurkara99/ISNAS-DIP.