# HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution

arXiv: https://arxiv.org/abs/2407.05878 · HTML: https://mlsanity.com/paper/2407.05878/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [DaViT: Dual Attention Vision Transformers](https://mlsanity.com/paper/2204.03645/) (2022)
- [Efficient Long-Range Attention Network for Image Super-resolution](https://mlsanity.com/paper/2203.06697/) (2022)
- [CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention](https://mlsanity.com/paper/2108.00154/) (2021)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [Residual Feature Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2009.11551/) (2020)

## Built on by (most cited first, 30 of 42)

- [The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2504.10686/) (2025)
- [Progressive Focused Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2503.20337/) (2025)
- [The Power of Context: How Multimodality Improves Image Super-Resolution](https://mlsanity.com/paper/2503.14503/) (2025)
- [NTIRE 2025 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2504.14582/) (2025)
- [Dual-domain Modulation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.10047/) (2025)
- [Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks](https://mlsanity.com/paper/2503.04223/) (2025)
- [C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales](https://mlsanity.com/paper/2503.13740/) (2025)
- [Training-Free Model Ensemble for Single-Image Super-Resolution via Strong-Branch Compensation](https://mlsanity.com/paper/2604.11564/) (2026)
- [Emulating Self-attention with Convolution for Efficient Image Super-Resolution](https://mlsanity.com/paper/2503.06671/) (2025)
- [DISTA-Net: Dynamic Closely-Spaced Infrared Small Target Unmixing](https://mlsanity.com/paper/2505.19148/) (2025)
- [PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors](https://mlsanity.com/paper/2506.02846/) (2025)
- [Boosting Edge Detection with Pixel-wise Feature Selection: The Extractor-Selector Paradigm](https://mlsanity.com/paper/2501.02534/) (2025)
- [FourierSR: A Fourier Token-based Plugin for Efficient Image Super-Resolution](https://mlsanity.com/paper/2503.10043/) (2025)
- [RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions](https://mlsanity.com/paper/2504.07603/) (2025)
- [Fast and Accurate Image Restoration and Generation with Rank Enhanced Linear Attention](https://mlsanity.com/paper/2505.16157/) (2025)
- [Multi-Scale Local Speculative Decoding for Image Generation](https://mlsanity.com/paper/2601.05149/) (2026)
- [F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal](https://mlsanity.com/paper/2506.05489/) (2025)
- [FluxFlow: Conservative Flow-Matching for Astronomical Image Super-Resolution](https://mlsanity.com/paper/2605.03749/) (2026)
- [MoCoLSK: Modality Conditioned High-Resolution Downscaling for Land Surface Temperature](https://mlsanity.com/paper/2409.19835/) (2024)
- [Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2506.11823/) (2025)
- [Your Super Resolution Model is not Enough for Tackling Real-World Scenarios](https://mlsanity.com/paper/2509.06387/) (2025)
- [Transformer-Progressive Mamba Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2511.03232/) (2025)
- [PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents](https://mlsanity.com/paper/2605.10341/) (2026)
- [EchoSR: Efficient Context Harnessing for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2605.17470/) (2026)
- [OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolution](https://mlsanity.com/paper/2609.06490/) (2026)
- [Unifying Dimensions: A Linear Adaptive Approach to Lightweight Image Super-Resolution](https://mlsanity.com/paper/2409.17597/) (2024)
- [Optimization of Module Transferability in Single Image Super-Resolution: Universality Assessment and Cycle Residual Blocks](https://mlsanity.com/paper/2505.03522/) (2025)
- [Dataset Distillation for Super-Resolution without Class Labels and Pre-trained Models](https://mlsanity.com/paper/2509.14777/) (2025)
- [HSNet: Heterogeneous Subgraph Network for Single Image Super-resolution](https://mlsanity.com/paper/2510.06564/) (2025)
- [HDW-SR: High-Frequency Guided Diffusion Model based on Wavelet Decomposition for Image Super-Resolution](https://mlsanity.com/paper/2511.13175/) (2025)

## Abstract (arXiv)

Transformers have exhibited promising performance in computer vision tasks including image super-resolution (SR). However, popular transformer-based SR methods often employ window self-attention with quadratic computational complexity to window sizes, resulting in fixed small windows with limited receptive fields. In this paper, we present a general strategy to convert transformer-based SR networks to hierarchical transformers (HiT-SR), boosting SR performance with multi-scale features while maintaining an efficient design. Specifically, we first replace the commonly used fixed small windows with expanding hierarchical windows to aggregate features at different scales and establish long-range dependencies. Considering the intensive computation required for large windows, we further design a spatial-channel correlation method with linear complexity to window sizes, efficiently gathering spatial and channel information from hierarchical windows. Extensive experiments verify the effectiveness and efficiency of our HiT-SR, and our improved versions of SwinIR-Light, SwinIR-NG, and SRFormer-Light yield state-of-the-art SR results with fewer parameters, FLOPs, and faster speeds (~7×).