# Improving Image Restoration by Revisiting Global Information Aggregation

arXiv: https://arxiv.org/abs/2112.04491 · HTML: https://mlsanity.com/paper/2112.04491/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 69 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 30 of 48)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Multi-level Wavelet-CNN for Image Restoration](https://mlsanity.com/paper/1805.07071/) (2018)

## Built on by (most cited first, 30 of 61)

- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [A Comparative Study of Image Restoration Networks for General Backbone Network Design](https://mlsanity.com/paper/2310.11881/) (2023)
- [The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2404.10343/) (2024)
- [Efficient Visual State Space Model for Image Deblurring](https://mlsanity.com/paper/2405.14343/) (2024)
- [OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer](https://mlsanity.com/paper/2302.03453/) (2023)
- [NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results](https://mlsanity.com/paper/2404.14248/) (2024)
- [AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring](https://mlsanity.com/paper/2406.09135/) (2024)
- [DiffI2I: Efficient Diffusion Model for Image-to-Image Translation](https://mlsanity.com/paper/2308.13767/) (2023)
- [Prompt-based Ingredient-Oriented All-in-One Image Restoration](https://mlsanity.com/paper/2309.03063/) (2023)
- [NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2504.12711/) (2025)
- [The Tenth NTIRE 2025 Image Denoising Challenge Report](https://mlsanity.com/paper/2504.12276/) (2025)
- [AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results](https://mlsanity.com/paper/2208.11184/) (2022)
- [Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring](https://mlsanity.com/paper/2404.13153/) (2024)
- [LoFormer: Local Frequency Transformer for Image Deblurring](https://mlsanity.com/paper/2407.16993/) (2024)
- [DIPNet: Efficiency Distillation and Iterative Pruning for Image Super-Resolution](https://mlsanity.com/paper/2304.07018/) (2023)
- [Multi-scale frequency separation network for image deblurring](https://mlsanity.com/paper/2206.00798/) (2022)
- [CascadedGaze: Efficiency in Global Context Extraction for Image Restoration](https://mlsanity.com/paper/2401.15235/) (2024)
- [Mixed Hierarchy Network for Image Restoration](https://mlsanity.com/paper/2302.09554/) (2023)
- [NTIRE 2025 Challenge on Efficient Burst HDR and Restoration: Datasets, Methods, and Results](https://mlsanity.com/paper/2505.12089/) (2025)
- [DLGSANet: Lightweight Dynamic Local and Global Self-Attention Networks for Image Super-Resolution](https://mlsanity.com/paper/2301.02031/) (2023)
- [Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers](https://mlsanity.com/paper/2306.12992/) (2023)
- [Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report](https://mlsanity.com/paper/2210.11153/) (2022)
- [SAM-Deblur: Let Segment Anything Boost Image Deblurring](https://mlsanity.com/paper/2309.02270/) (2023)
- [Revisiting Image Deblurring with an Efficient ConvNet](https://mlsanity.com/paper/2302.02234/) (2023)
- [Dual-domain Modulation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.10047/) (2025)
- [CLIP-Guided Data Augmentation for Night-Time Image Dehazing](https://mlsanity.com/paper/2604.05500/) (2026)
- [Partial Large Kernel CNNs for Efficient Super-Resolution](https://mlsanity.com/paper/2404.11848/) (2024)
- [Dual-Branch Remote Sensing Infrared Image Super-Resolution](https://mlsanity.com/paper/2604.10112/) (2026)
- [A Preliminary Exploration Towards General Image Restoration](https://mlsanity.com/paper/2408.15143/) (2024)

## Abstract (arXiv)

Global operations, such as global average pooling, are widely used in top-performance image restorers. They aggregate global information from input features along entire spatial dimensions but behave differently during training and inference in image restoration tasks: they are based on different regions, namely the cropped patches (from images) and the full-resolution images. This paper revisits global information aggregation and finds that the image-based features during inference have a different distribution than the patch-based features during training. This train-test inconsistency negatively impacts the performance of models, which is severely overlooked by previous works. To reduce the inconsistency and improve test-time performance, we propose a simple method called Test-time Local Converter (TLC). Our TLC converts global operations to local ones only during inference so that they aggregate features within local spatial regions rather than the entire large images. The proposed method can be applied to various global modules (e.g., normalization, channel and spatial attention) with negligible costs. Without the need for any fine-tuning, TLC improves state-of-the-art results on several image restoration tasks, including single-image motion deblurring, video deblurring, defocus deblurring, and image denoising. In particular, with TLC, our Restormer-Local improves the state-of-the-art result in single image deblurring from 32.92 dB to 33.57 dB on GoPro dataset. The code is available at https://github.com/megvii-research/tlc.