# Unfolding Taylor's Approximations for Image Restoration

arXiv: https://arxiv.org/abs/2109.03442 · HTML: https://mlsanity.com/paper/2109.03442/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 79 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Human-Aware Motion Deblurring](https://mlsanity.com/paper/2001.06816/) (2020)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring](https://mlsanity.com/paper/2004.05343/) (2020)
- [Multi-Temporal Recurrent Neural Networks For Progressive Non-Uniform Single Image Deblurring With Incremental Temporal Training](https://mlsanity.com/paper/1911.07410/) (2019)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur](https://mlsanity.com/paper/1612.02583/) (2016)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [A Neural Approach to Blind Motion Deblurring](https://mlsanity.com/paper/1603.04771/) (2016)
- [Lightweight Pyramid Networks for Image Deraining](https://mlsanity.com/paper/1805.06173/) (2018)
- [DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal](https://mlsanity.com/paper/2008.00767/) (2020)
- [Graph-Based Blind Image Deblurring From a Single Photograph](https://mlsanity.com/paper/1802.07929/) (2018)
- [Neural Sparse Representation for Image Restoration](https://mlsanity.com/paper/2006.04357/) (2020)
- [Physical Model Guided Deep Image Deraining](https://mlsanity.com/paper/2003.13242/) (2020)

## Built on by (most cited first, 6 of 6)

- [Deep Fourier Up-Sampling](https://mlsanity.com/paper/2210.05171/) (2022)
- [Single UHD Image Dehazing via Interpretable Pyramid Network](https://mlsanity.com/paper/2202.08589/) (2022)
- [Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising](https://mlsanity.com/paper/2208.11803/) (2022)
- [Decomposition Ascribed Synergistic Learning for Unified Image Restoration](https://mlsanity.com/paper/2308.00759/) (2023)
- [Random Weights Networks Work as Loss Prior Constraint for Image Restoration](https://mlsanity.com/paper/2303.16438/) (2023)
- [Three-Stage Cascade Framework for Blurry Video Frame Interpolation](https://mlsanity.com/paper/2310.05383/) (2023)

## Abstract (arXiv)

Deep learning provides a new avenue for image restoration, which demands a delicate balance between fine-grained details and high-level contextualized information during recovering the latent clear image. In practice, however, existing methods empirically construct encapsulated end-to-end mapping networks without deepening into the rationality, and neglect the intrinsic prior knowledge of restoration task. To solve the above problems, inspired by Taylor's Approximations, we unfold Taylor's Formula to construct a novel framework for image restoration. We find the main part and the derivative part of Taylor's Approximations take the same effect as the two competing goals of high-level contextualized information and spatial details of image restoration respectively. Specifically, our framework consists of two steps, correspondingly responsible for the mapping and derivative functions. The former first learns the high-level contextualized information and the later combines it with the degraded input to progressively recover local high-order spatial details. Our proposed framework is orthogonal to existing methods and thus can be easily integrated with them for further improvement, and extensive experiments demonstrate the effectiveness and scalability of our proposed framework.