# GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions

arXiv: https://arxiv.org/abs/2305.17863 · HTML: https://mlsanity.com/paper/2305.17863/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 104 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 30 of 56)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Multi-Scale Boosted Dehazing Network with Dense Feature Fusion](https://mlsanity.com/paper/2004.13388/) (2020)

## Built on by (most cited first, 30 of 62)

- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [MB-TaylorFormer: Multi-branch Efficient Transformer Expanded by Taylor Formula for Image Dehazing](https://mlsanity.com/paper/2308.14036/) (2023)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [Restoring Images in Adverse Weather Conditions via Histogram Transformer](https://mlsanity.com/paper/2407.10172/) (2024)
- [Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration](https://mlsanity.com/paper/2408.15994/) (2024)
- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks](https://mlsanity.com/paper/2503.16930/) (2025)
- [MB-TaylorFormer V2: Improved Multi-branch Linear Transformer Expanded by Taylor Formula for Image Restoration](https://mlsanity.com/paper/2501.04486/) (2025)
- [Learning A Physical-aware Diffusion Model Based on Transformer for Underwater Image Enhancement](https://mlsanity.com/paper/2403.01497/) (2024)
- [From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspective](https://mlsanity.com/paper/2503.13165/) (2025)
- [Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal](https://mlsanity.com/paper/2310.18293/) (2023)
- [RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization](https://mlsanity.com/paper/2402.05628/) (2024)
- [CFMW: Cross-modality Fusion Mamba for Robust Object Detection under Adverse Weather](https://mlsanity.com/paper/2404.16302/) (2024)
- [Efficient Degradation-agnostic Image Restoration via Channel-Wise Functional Decomposition and Manifold Regularization](https://mlsanity.com/paper/2505.18679/) (2025)
- [DSwinIR: Rethinking Window-based Attention for Image Restoration](https://mlsanity.com/paper/2504.04869/) (2025)
- [M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration](https://mlsanity.com/paper/2506.07814/) (2025)
- [EvoIR: Towards All-in-One Image Restoration via Evolutionary Frequency Modulation](https://mlsanity.com/paper/2512.05104/) (2025)
- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration](https://mlsanity.com/paper/2504.09973/) (2025)
- [Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration](https://mlsanity.com/paper/2410.04811/) (2024)
- [ClusIR: Towards Cluster-Guided All-in-One Image Restoration](https://mlsanity.com/paper/2512.10948/) (2025)
- [Robust Physics-based Deep MRI Reconstruction Via Diffusion Purification](https://mlsanity.com/paper/2309.05794/) (2023)
- [Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal](https://mlsanity.com/paper/2503.09013/) (2025)
- [Learning Where and What to Restore for Composite Image Restoration](https://mlsanity.com/paper/2503.17915/) (2025)
- [SoftHGNN: Soft Hypergraph Neural Networks for General Visual Recognition](https://mlsanity.com/paper/2505.15325/) (2025)
- [Benchmarking Ultra-High-Definition Image Reflection Removal](https://mlsanity.com/paper/2308.00265/) (2023)
- [Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinement](https://mlsanity.com/paper/2510.07961/) (2025)
- [UDPNet: Unleashing Depth-based Priors for Robust Image Dehazing](https://mlsanity.com/paper/2601.06909/) (2026)
- [PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.17055/) (2026)
- [A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting](https://mlsanity.com/paper/2604.05629/) (2026)

## Abstract (arXiv)

Image restoration in adverse weather conditions is a difficult task in computer vision. In this paper, we propose a novel transformer-based framework called GridFormer which serves as a backbone for image restoration under adverse weather conditions. GridFormer is designed in a grid structure using a residual dense transformer block, and it introduces two core designs. First, it uses an enhanced attention mechanism in the transformer layer. The mechanism includes stages of the sampler and compact self-attention to improve efficiency, and a local enhancement stage to strengthen local information. Second, we introduce a residual dense transformer block (RDTB) as the final GridFormer layer. This design further improves the network's ability to learn effective features from both preceding and current local features. The GridFormer framework achieves state-of-the-art results on five diverse image restoration tasks in adverse weather conditions, including image deraining, dehazing, deraining & dehazing, desnowing, and multi-weather restoration. The source code and pre-trained models are available at https://github.com/TaoWangzj/GridFormer.