# Model-Guided Network with Cluster-Based Operators for Spatio-Spectral Super-Resolution

arXiv: https://arxiv.org/abs/2505.24605 · HTML: https://mlsanity.com/paper/2505.24605/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 75 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Efficient Long-Range Attention Network for Image Super-resolution](https://mlsanity.com/paper/2203.06697/) (2022)
- [Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery](https://mlsanity.com/paper/2005.08752/) (2020)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [Fusformer: A Transformer-based Fusion Approach for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2109.02079/) (2021)
- [NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image](https://mlsanity.com/paper/2005.03412/) (2020)
- [AdaptiveWeighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images](https://mlsanity.com/paper/2005.09305/) (2020)
- [ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2307.14010/) (2023)
- [Learned Spectral Super-Resolution](https://mlsanity.com/paper/1703.09470/) (2017)
- [PanFormer: a Transformer Based Model for Pan-sharpening](https://mlsanity.com/paper/2203.02916/) (2022)
- [Remote Sensing Image Super-resolution and Object Detection: Benchmark and State of the Art](https://mlsanity.com/paper/2111.03260/) (2021)
- [Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB](https://mlsanity.com/paper/1709.00265/) (2017)
- [Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN](https://mlsanity.com/paper/1806.03575/) (2018)
- [Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images](https://mlsanity.com/paper/2108.05547/) (2021)
- [A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model](https://mlsanity.com/paper/1606.05703/) (2016)

## Abstract (arXiv)

This paper addresses the problem of reconstructing a high-resolution hyperspectral image from a low-resolution multispectral observation. While spatial super-resolution and spectral super-resolution have been extensively studied, joint spatio-spectral super-resolution remains relatively explored. We propose an end-to-end model-driven framework that explicitly decomposes the joint spatio-spectral super-resolution problem into spatial super-resolution, spectral super-resolution and fusion tasks. Each sub-task is addressed by unfolding a variational-based approach, where the operators involved in the proximal gradient iterative scheme are replaced with tailored learnable modules. In particular, we design an upsampling operator for spatial super-resolution based on classical back-projection algorithms, adapted to handle arbitrary scaling factors. Spectral reconstruction is performed using learnable cluster-based upsampling and downsampling operators. For image fusion, we integrate low-frequency estimation and high-frequency injection modules to combine the spatial and spectral information from spatial super-resolution and spectral super-resolution outputs. Additionally, we introduce an efficient nonlocal post-processing step that leverages image self-similarity by combining a multi-head attention mechanism with residual connections. Extensive evaluations on several datasets and sampling factors demonstrate the effectiveness of our approach. The source code will be available at https://github.com/TAMI-UIB/JSSUNet