# Attention in Attention Network for Image Super-Resolution

arXiv: https://arxiv.org/abs/2104.09497 · HTML: https://mlsanity.com/paper/2104.09497/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 30 of 30)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [Channel-wise and Spatial Feature Modulation Network for Single Image Super-Resolution](https://mlsanity.com/paper/1809.11130/) (2018)
- [Lightweight Image Super-Resolution with Adaptive Weighted Learning Network](https://mlsanity.com/paper/1904.02358/) (2019)
- [Lightweight Single-Image Super-Resolution Network with Attentive Auxiliary Feature Learning](https://mlsanity.com/paper/2011.06773/) (2020)
- [Hierarchical Residual Attention Network for Single Image Super-Resolution](https://mlsanity.com/paper/2012.04578/) (2020)

## Built on by (most cited first, 20 of 20)

- [Dual Aggregation Transformer for Image Super-Resolution](https://mlsanity.com/paper/2308.03364/) (2023)
- [Cross Aggregation Transformer for Image Restoration](https://mlsanity.com/paper/2211.13654/) (2022)
- [Accurate Image Restoration with Attention Retractable Transformer](https://mlsanity.com/paper/2210.01427/) (2022)
- [Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks](https://mlsanity.com/paper/2308.14153/) (2023)
- [Blueprint Separable Residual Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2205.05996/) (2022)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning](https://mlsanity.com/paper/2403.02601/) (2024)
- [Efficient Image Super-Resolution using Vast-Receptive-Field Attention](https://mlsanity.com/paper/2210.05960/) (2022)
- [The Fourth Challenge on Image Super-Resolution (×4) at NTIRE 2026: Benchmark Results and Method Overview](https://mlsanity.com/paper/2604.14558/) (2026)
- [Adaptive Visual Imitation Learning for Robotic Assisted Feeding Across Varied Bowl Configurations and Food Types](https://mlsanity.com/paper/2403.12891/) (2024)
- [A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators](https://mlsanity.com/paper/2112.10046/) (2021)
- [Local-Selective Feature Distillation for Single Image Super-Resolution](https://mlsanity.com/paper/2111.10988/) (2021)
- [Efficient Mixed Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2305.11403/) (2023)
- [Super-Resolution by Predicting Offsets: An Ultra-Efficient Super-Resolution Network for Rasterized Images](https://mlsanity.com/paper/2210.04198/) (2022)
- [Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2407.18574/) (2024)
- [Multi-Convformer: Extending Conformer with Multiple Convolution Kernels](https://mlsanity.com/paper/2407.03718/) (2024)
- [HASN: Hybrid Attention Separable Network for Efficient Image Super-resolution](https://mlsanity.com/paper/2410.09844/) (2024)
- [DmADs-Net: Dense multiscale attention and depth-supervised network for medical image segmentation](https://mlsanity.com/paper/2405.00472/) (2024)
- [Embedding Similarity Guided License Plate Super Resolution](https://mlsanity.com/paper/2501.01483/) (2025)
- [Not All Degradations Are Equal: A Targeted Feature Denoising Framework for Generalizable Image Super-Resolution](https://mlsanity.com/paper/2509.14841/) (2025)

## Abstract (arXiv)

Convolutional neural networks have allowed remarkable advances in single image super-resolution (SISR) over the last decade. Among recent advances in SISR, attention mechanisms are crucial for high-performance SR models. However, the attention mechanism remains unclear on why and how it works in SISR. In this work, we attempt to quantify and visualize attention mechanisms in SISR and show that not all attention modules are equally beneficial. We then propose attention in attention network (A2N) for more efficient and accurate SISR. Specifically, A2N consists of a non-attention branch and a coupling attention branch. A dynamic attention module is proposed to generate weights for these two branches to suppress unwanted attention adjustments dynamically, where the weights change adaptively according to the input features. This allows attention modules to specialize to beneficial examples without otherwise penalties and thus greatly improve the capacity of the attention network with few parameters overhead. Experimental results demonstrate that our final model A2N could achieve superior trade-off performances comparing with state-of-the-art networks of similar sizes. Codes are available at https://github.com/haoyuc/A2N.