# Cross Aggregation Transformer for Image Restoration

arXiv: https://arxiv.org/abs/2211.13654 · HTML: https://mlsanity.com/paper/2211.13654/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Conditional Positional Encodings for Vision Transformers](https://mlsanity.com/paper/2102.10882/) (2021)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [Incorporating Convolution Designs into Visual Transformers](https://mlsanity.com/paper/2103.11816/) (2021)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Learning Enriched Features for Fast Image Restoration and Enhancement](https://mlsanity.com/paper/2205.01649/) (2022)
- [CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention](https://mlsanity.com/paper/2108.00154/) (2021)

## Built on by (most cited first, 30 of 92)

- [Dual Aggregation Transformer for Image Super-Resolution](https://mlsanity.com/paper/2308.03364/) (2023)
- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary](https://mlsanity.com/paper/2401.08209/) (2024)
- [Masked Image Training for Generalizable Deep Image Denoising](https://mlsanity.com/paper/2303.13132/) (2023)
- [DRCT: Saving Image Super-resolution away from Information Bottleneck](https://mlsanity.com/paper/2404.00722/) (2024)
- [NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2404.09790/) (2024)
- [A Survey of Camouflaged Object Detection and Beyond](https://mlsanity.com/paper/2408.14562/) (2024)
- [Recursive Generalization Transformer for Image Super-Resolution](https://mlsanity.com/paper/2303.06373/) (2023)
- [Progressive Focused Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2503.20337/) (2025)
- [ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2307.14010/) (2023)
- [Image Super-Resolution with Text Prompt Diffusion](https://mlsanity.com/paper/2311.14282/) (2023)
- [Crafting Training Degradation Distribution for the Accuracy-Generalization Trade-off in Real-World Super-Resolution](https://mlsanity.com/paper/2305.18107/) (2023)
- [Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator](https://mlsanity.com/paper/2410.04224/) (2024)
- [Dropout the High-rate Downsampling: A Novel Design Paradigm for UHD Image Restoration](https://mlsanity.com/paper/2411.06456/) (2024)
- [Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration](https://mlsanity.com/paper/2404.00288/) (2024)
- [Binarized Diffusion Model for Image Super-Resolution](https://mlsanity.com/paper/2406.05723/) (2024)
- [CascadedGaze: Efficiency in Global Context Extraction for Image Restoration](https://mlsanity.com/paper/2401.15235/) (2024)
- [CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.06896/) (2025)
- [Networks are Slacking Off: Understanding Generalization Problem in Image Deraining](https://mlsanity.com/paper/2305.15134/) (2023)
- [Sharing Key Semantics in Transformer Makes Efficient Image Restoration](https://mlsanity.com/paper/2405.20008/) (2024)
- [2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution](https://mlsanity.com/paper/2406.06649/) (2024)
- [Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach](https://mlsanity.com/paper/2401.05633/) (2024)
- [HAIR: Hypernetworks-based All-in-One Image Restoration](https://mlsanity.com/paper/2408.08091/) (2024)
- [WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal](https://mlsanity.com/paper/2303.13739/) (2023)
- [Efficient Degradation-aware Any Image Restoration](https://mlsanity.com/paper/2405.15475/) (2024)
- [NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Study](https://mlsanity.com/paper/2504.15003/) (2025)
- [Exploring Frequency-Inspired Optimization in Transformer for Efficient Single Image Super-Resolution](https://mlsanity.com/paper/2308.05022/) (2023)
- [Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2311.17340/) (2023)
- [Training Transformer Models by Wavelet Losses Improves Quantitative and Visual Performance in Single Image Super-Resolution](https://mlsanity.com/paper/2404.11273/) (2024)

## Abstract (arXiv)

Recently, Transformer architecture has been introduced into image restoration to replace convolution neural network (CNN) with surprising results. Considering the high computational complexity of Transformer with global attention, some methods use the local square window to limit the scope of self-attention. However, these methods lack direct interaction among different windows, which limits the establishment of long-range dependencies. To address the above issue, we propose a new image restoration model, Cross Aggregation Transformer (CAT). The core of our CAT is the Rectangle-Window Self-Attention (Rwin-SA), which utilizes horizontal and vertical rectangle window attention in different heads parallelly to expand the attention area and aggregate the features cross different windows. We also introduce the Axial-Shift operation for different window interactions. Furthermore, we propose the Locality Complementary Module to complement the self-attention mechanism, which incorporates the inductive bias of CNN (e.g., translation invariance and locality) into Transformer, enabling global-local coupling. Extensive experiments demonstrate that our CAT outperforms recent state-of-the-art methods on several image restoration applications. The code and models are available at https://github.com/zhengchen1999/CAT.