# ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution

arXiv: https://arxiv.org/abs/2307.14010 · HTML: https://mlsanity.com/paper/2307.14010/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 73 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [LocalViT: Analyzing Locality in Vision Transformers](https://mlsanity.com/paper/2104.05707/) (2021)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Efficient Long-Range Attention Network for Image Super-resolution](https://mlsanity.com/paper/2203.06697/) (2022)
- [ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond](https://mlsanity.com/paper/2202.10108/) (2022)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Cross Aggregation Transformer for Image Restoration](https://mlsanity.com/paper/2211.13654/) (2022)
- [Is Attention Better Than Matrix Decomposition?](https://mlsanity.com/paper/2109.04553/) (2021)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [Residual Local Feature Network for Efficient Super-Resolution](https://mlsanity.com/paper/2205.07514/) (2022)
- [Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding](https://mlsanity.com/paper/2106.12566/) (2021)
- [Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery](https://mlsanity.com/paper/2005.08752/) (2020)
- [Vision Transformer with Quadrangle Attention](https://mlsanity.com/paper/2303.15105/) (2023)

## Built on by (most cited first, 23 of 23)

- [IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection](https://mlsanity.com/paper/2407.07520/) (2024)
- [HSIGene: A Foundation Model For Hyperspectral Image Generation](https://mlsanity.com/paper/2409.12470/) (2024)
- [MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration](https://mlsanity.com/paper/2503.09131/) (2025)
- [Multimodal Informative ViT: Information Aggregation and Distribution for Hyperspectral and LiDAR Classification](https://mlsanity.com/paper/2401.03179/) (2024)
- [I3Net: Inter-Intra-slice Interpolation Network for Medical Slice Synthesis](https://mlsanity.com/paper/2405.02857/) (2024)
- [Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning](https://mlsanity.com/paper/2603.07918/) (2026)
- [AI-Driven HSI: Multimodality, Fusion, Challenges, and the Deep Learning Revolution](https://mlsanity.com/paper/2502.06894/) (2025)
- [Multispectral Demosaicing via Dual Cameras](https://mlsanity.com/paper/2503.22026/) (2025)
- [Onboard Hyperspectral Super-Resolution with Deep Pushbroom Neural Network](https://mlsanity.com/paper/2507.20765/) (2025)
- [Degradation-Aware Metric Prompting for Hyperspectral Image Restoration](https://mlsanity.com/paper/2512.20251/) (2025)
- [Cascaded Multi-Scale Attention for Enhanced Multi-Scale Feature Extraction and Interaction with Low-Resolution Images](https://mlsanity.com/paper/2412.02197/) (2024)
- [Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity](https://mlsanity.com/paper/2505.11267/) (2025)
- [Model-Guided Network with Cluster-Based Operators for Spatio-Spectral Super-Resolution](https://mlsanity.com/paper/2505.24605/) (2025)
- [Controllable Reference Guided Diffusion with Local Global Fusion for Real World Remote Sensing Image Super Resolution](https://mlsanity.com/paper/2506.23801/) (2025)
- [MambaX: Image Super-Resolution with State Predictive Control](https://mlsanity.com/paper/2511.18028/) (2025)
- [DCCS-Det: Directional Context and Cross-Scale-Aware Detector for Infrared Small Target](https://mlsanity.com/paper/2601.16428/) (2026)
- [Correcting Spectra Outside the Backbone: A Model-Agnostic Rectifier for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2601.21338/) (2026)
- [CoFusion: Multispectral and Hyperspectral Image Fusion via Spectral Coordinate Attention](https://mlsanity.com/paper/2604.10584/) (2026)
- [Self-Supervised Super-Resolution for Sentinel-5P Hyperspectral Images](https://mlsanity.com/paper/2604.17652/) (2026)
- [Spectral Dynamic Attention Network for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2604.27326/) (2026)
- [R2H-Diff: Guided Spectral Diffusion Model for RGB-to-Hyperspectral Reconstruction](https://mlsanity.com/paper/2605.05688/) (2026)
- [HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation](https://mlsanity.com/paper/2605.13581/) (2026)
- [Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators](https://mlsanity.com/paper/2609.39926/) (2026)

## Abstract (arXiv)

Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. However, the prevailing CNN-based approaches have shown limitations in building long-range dependencies and capturing interaction information between spectral features. This results in inadequate utilization of spectral information and artifacts after upsampling. To address this issue, we propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure. Specifically, we first introduce a robust and spectral-friendly similarity metric, ie, the spectral correlation coefficient of the spectrum (SCC), to replace the original attention matrix and incorporates inductive biases into the model to facilitate training. Built upon it, we further utilize the kernelizable attention technique with theoretical support to form a novel efficient SCC-kernel-based self-attention (ESSA) and reduce attention computation to linear complexity. ESSA enlarges the receptive field for features after upsampling without bringing much computation and allows the model to effectively utilize spatial-spectral information from different scales, resulting in the generation of more natural high-resolution images. Without the need for pretraining on large-scale datasets, our experiments demonstrate ESSA's effectiveness in both visual quality and quantitative results.