# Spatially-Adaptive Image Restoration using Distortion-Guided Networks

arXiv: https://arxiv.org/abs/2108.08617 · HTML: https://mlsanity.com/paper/2108.08617/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 73 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 30 of 47)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [BlockDrop: Dynamic Inference Paths in Residual Networks](https://mlsanity.com/paper/1711.08393/) (2017)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Human-Aware Motion Deblurring](https://mlsanity.com/paper/2001.06816/) (2020)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)

## Built on by (most cited first, 30 of 77)

- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Efficient and Explicit Modelling of Image Hierarchies for Image Restoration](https://mlsanity.com/paper/2303.00748/) (2023)
- [Stripformer: Strip Transformer for Fast Image Deblurring](https://mlsanity.com/paper/2204.04627/) (2022)
- [Event-Based Fusion for Motion Deblurring with Cross-modal Attention](https://mlsanity.com/paper/2112.00167/) (2021)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [VmambaIR: Visual State Space Model for Image Restoration](https://mlsanity.com/paper/2403.11423/) (2024)
- [Hierarchical Integration Diffusion Model for Realistic Image Deblurring](https://mlsanity.com/paper/2305.12966/) (2023)
- [Intriguing Findings of Frequency Selection for Image Deblurring](https://mlsanity.com/paper/2111.11745/) (2021)
- [Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Model](https://mlsanity.com/paper/2403.11157/) (2024)
- [Improving Image Restoration by Revisiting Global Information Aggregation](https://mlsanity.com/paper/2112.04491/) (2021)
- [A Comparative Study of Image Restoration Networks for General Backbone Network Design](https://mlsanity.com/paper/2310.11881/) (2023)
- [FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining](https://mlsanity.com/paper/2404.09476/) (2024)
- [Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks](https://mlsanity.com/paper/2308.14153/) (2023)
- [Improving Image Restoration through Removing Degradations in Textual Representations](https://mlsanity.com/paper/2312.17334/) (2023)
- [MB-TaylorFormer V2: Improved Multi-branch Linear Transformer Expanded by Taylor Formula for Image Restoration](https://mlsanity.com/paper/2501.04486/) (2025)
- [Towards Unified Deep Image Deraining: A Survey and A New Benchmark](https://mlsanity.com/paper/2310.03535/) (2023)
- [KBNet: Kernel Basis Network for Image Restoration](https://mlsanity.com/paper/2303.02881/) (2023)
- [FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining](https://mlsanity.com/paper/2405.19450/) (2024)
- [CU-Mamba: Selective State Space Models with Channel Learning for Image Restoration](https://mlsanity.com/paper/2404.11778/) (2024)
- [Prompt-based Ingredient-Oriented All-in-One Image Restoration](https://mlsanity.com/paper/2309.03063/) (2023)
- [Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring](https://mlsanity.com/paper/2404.13153/) (2024)
- [Multi-scale frequency separation network for image deblurring](https://mlsanity.com/paper/2206.00798/) (2022)
- [SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolution](https://mlsanity.com/paper/2303.09735/) (2023)
- [Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration](https://mlsanity.com/paper/2404.00288/) (2024)
- [CascadedGaze: Efficiency in Global Context Extraction for Image Restoration](https://mlsanity.com/paper/2401.15235/) (2024)
- [Mixed Hierarchy Network for Image Restoration](https://mlsanity.com/paper/2302.09554/) (2023)
- [Sharing Key Semantics in Transformer Makes Efficient Image Restoration](https://mlsanity.com/paper/2405.20008/) (2024)
- [HAIR: Hypernetworks-based All-in-One Image Restoration](https://mlsanity.com/paper/2408.08091/) (2024)
- [Efficient Degradation-aware Any Image Restoration](https://mlsanity.com/paper/2405.15475/) (2024)
- [Intra and Inter Parser-Prompted Transformers for Effective Image Restoration](https://mlsanity.com/paper/2503.14037/) (2025)

## Abstract (arXiv)

We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regions in the image. SPAIR comprises of two components, (1) a localization network that identifies degraded pixels, and (2) a restoration network that exploits knowledge from the localization network in filter and feature domain to selectively and adaptively restore degraded pixels. Our key idea is to exploit the non-uniformity of heavy degradations in spatial-domain and suitably embed this knowledge within distortion-guided modules performing sparse normalization, feature extraction and attention. Our architecture is agnostic to physical formation model and generalizes across several types of spatially-varying degradations. We demonstrate the efficacy of SPAIR individually on four restoration tasks-removal of rain-streaks, raindrops, shadows and motion blur. Extensive qualitative and quantitative comparisons with prior art on 11 benchmark datasets demonstrate that our degradation-agnostic network design offers significant performance gains over state-of-the-art degradation-specific architectures. Code available at https://github.com/human-analysis/spatially-adaptive-image-restoration.