# N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution

arXiv: https://arxiv.org/abs/2211.11436 · HTML: https://mlsanity.com/paper/2211.11436/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 93 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)

## Built on by (most cited first, 30 of 46)

- [Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary](https://mlsanity.com/paper/2401.08209/) (2024)
- [The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2404.10343/) (2024)
- [HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution](https://mlsanity.com/paper/2407.05878/) (2024)
- [Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution](https://mlsanity.com/paper/2405.07023/) (2024)
- [CAMixerSR: Only Details Need More "Attention"](https://mlsanity.com/paper/2402.19289/) (2024)
- [Progressive Focused Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2503.20337/) (2025)
- [Swift Parameter-free Attention Network for Efficient Super-Resolution](https://mlsanity.com/paper/2311.12770/) (2023)
- [See More Details: Efficient Image Super-Resolution by Experts Mining](https://mlsanity.com/paper/2402.03412/) (2024)
- [Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Network](https://mlsanity.com/paper/2212.14181/) (2022)
- [Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach](https://mlsanity.com/paper/2401.05633/) (2024)
- [Dual-domain Modulation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.10047/) (2025)
- [Partial Large Kernel CNNs for Efficient Super-Resolution](https://mlsanity.com/paper/2404.11848/) (2024)
- [Reciprocal Attention Mixing Transformer for Lightweight Image Restoration](https://mlsanity.com/paper/2305.11474/) (2023)
- [Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks](https://mlsanity.com/paper/2503.04223/) (2025)
- [C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales](https://mlsanity.com/paper/2503.13740/) (2025)
- [Training-Free Model Ensemble for Single-Image Super-Resolution via Strong-Branch Compensation](https://mlsanity.com/paper/2604.11564/) (2026)
- [MAT: Multi-Range Attention Transformer for Efficient Image Super-Resolution](https://mlsanity.com/paper/2411.17214/) (2024)
- [Efficient Mixed Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2305.11403/) (2023)
- [PromptSR: Cascade Prompting for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2507.04118/) (2025)
- [Event Camera Demosaicing via Swin Transformer and Pixel-focus Loss](https://mlsanity.com/paper/2404.02731/) (2024)
- [LIPT: Latency-aware Image Processing Transformer](https://mlsanity.com/paper/2404.06075/) (2024)
- [FourierSR: A Fourier Token-based Plugin for Efficient Image Super-Resolution](https://mlsanity.com/paper/2503.10043/) (2025)
- [ATD: Improved Transformer with Adaptive Token Dictionary for Image Restoration](https://mlsanity.com/paper/2603.02581/) (2026)
- [Exploration of Lightweight Single Image Denoising with Transformers and Truly Fair Training](https://mlsanity.com/paper/2304.01805/) (2023)
- [Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2307.02974/) (2023)
- [Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition](https://mlsanity.com/paper/2404.19317/) (2024)
- [Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution](https://mlsanity.com/paper/2403.05808/) (2024)
- [Large coordinate kernel attention network for lightweight image super-resolution](https://mlsanity.com/paper/2405.09353/) (2024)
- [FFNet: MetaMixer-based Efficient Convolutional Mixer Design](https://mlsanity.com/paper/2406.02021/) (2024)
- [LMLT: Low-to-high Multi-Level Vision Transformer for Image Super-Resolution](https://mlsanity.com/paper/2409.03516/) (2024)

## Abstract (arXiv)

While some studies have proven that Swin Transformer (Swin) with window self-attention (WSA) is suitable for single image super-resolution (SR), the plain WSA ignores the broad regions when reconstructing high-resolution images due to a limited receptive field. In addition, many deep learning SR methods suffer from intensive computations. To address these problems, we introduce the N-Gram context to the low-level vision with Transformers for the first time. We define N-Gram as neighboring local windows in Swin, which differs from text analysis that views N-Gram as consecutive characters or words. N-Grams interact with each other by sliding-WSA, expanding the regions seen to restore degraded pixels. Using the N-Gram context, we propose NGswin, an efficient SR network with SCDP bottleneck taking multi-scale outputs of the hierarchical encoder. Experimental results show that NGswin achieves competitive performance while maintaining an efficient structure when compared with previous leading methods. Moreover, we also improve other Swin-based SR methods with the N-Gram context, thereby building an enhanced model: SwinIR-NG. Our improved SwinIR-NG outperforms the current best lightweight SR approaches and establishes state-of-the-art results. Codes are available at https://github.com/rami0205/NGramSwin.