# Random Weights Networks Work as Loss Prior Constraint for Image Restoration

arXiv: https://arxiv.org/abs/2303.16438 · HTML: https://mlsanity.com/paper/2303.16438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [Graph-Based Blind Image Deblurring From a Single Photograph](https://mlsanity.com/paper/1802.07929/) (2018)
- [Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement](https://mlsanity.com/paper/2112.05147/) (2021)
- [Neural Sparse Representation for Image Restoration](https://mlsanity.com/paper/2006.04357/) (2020)
- [GAN Prior based Null-Space Learning for Consistent Super-Resolution](https://mlsanity.com/paper/2211.13524/) (2022)
- [External Prior Guided Internal Prior Learning for Real-World Noisy Image Denoising](https://mlsanity.com/paper/1705.04505/) (2017)
- [Underexposed Image Correction via Hybrid Priors Navigated Deep Propagation](https://mlsanity.com/paper/1907.07408/) (2019)
- [Unfolding Taylor's Approximations for Image Restoration](https://mlsanity.com/paper/2109.03442/) (2021)

## Abstract (arXiv)

In this paper, orthogonal to the existing data and model studies, we instead resort our efforts to investigate the potential of loss function in a new perspective and present our belief ``Random Weights Networks can Be Acted as Loss Prior Constraint for Image Restoration''. Inspired by Functional theory, we provide several alternative solutions to implement our belief in the strict mathematical manifolds including Taylor's Unfolding Network, Invertible Neural Network, Central Difference Convolution and Zero-order Filtering as ``random weights network prototype'' with respect of the following four levels: 1) the different random weights strategies; 2) the different network architectures, eg, pure convolution layer or transformer; 3) the different network architecture depths; 4) the different numbers of random weights network combination. Furthermore, to enlarge the capability of the randomly initialized manifolds, we devise the manner of random weights in the following two variants: 1) the weights are randomly initialized only once during the whole training procedure; 2) the weights are randomly initialized at each training iteration epoch. Our propose belief can be directly inserted into existing networks without any training and testing computational cost. Extensive experiments across multiple image restoration tasks, including image de-noising, low-light image enhancement, guided image super-resolution demonstrate the consistent performance gains obtained by introducing our belief. To emphasize, our main focus is to spark the realms of loss function and save their current neglected status. Code will be publicly available.