# Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance

arXiv: https://arxiv.org/abs/2505.02109 · HTML: https://mlsanity.com/paper/2505.02109/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 81 references that are in the map.
- First posted on arXiv 4 May 2025.

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Robust Reference-based Super-Resolution via C2-Matching](https://mlsanity.com/paper/2106.01863/) (2021)
- [Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2203.04845/) (2022)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images](https://mlsanity.com/paper/2205.11733/) (2022)
- [Reference-based Image Super-Resolution with Deformable Attention Transformer](https://mlsanity.com/paper/2207.11938/) (2022)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery](https://mlsanity.com/paper/2005.08752/) (2020)
- [Instance Segmentation in the Dark](https://mlsanity.com/paper/2304.14298/) (2023)
- [HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models](https://mlsanity.com/paper/2306.12085/) (2023)
- [Learning optical flow from still images](https://mlsanity.com/paper/2104.03965/) (2021)
- [Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/1804.05042/) (2018)
- [Memory-augmented Deep Unfolding Network for Guided Image Super-resolution](https://mlsanity.com/paper/2203.04960/) (2022)
- [Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net](https://mlsanity.com/paper/1904.12175/) (2019)
- [Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance](https://mlsanity.com/paper/2302.06298/) (2023)
- [MPI-Flow: Learning Realistic Optical Flow with Multiplane Images](https://mlsanity.com/paper/2309.06714/) (2023)
- [Making the Invisible Visible: Toward High-Quality Terahertz Tomographic Imaging via Physics-Guided Restoration](https://mlsanity.com/paper/2304.14894/) (2023)

## Built on by (most cited first, 19 of 19)

- [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025)
- [Real Noise Decoupling for Hyperspectral Image Denoising](https://mlsanity.com/paper/2511.17196/) (2025)
- [Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration](https://mlsanity.com/paper/2602.11545/) (2026)
- [Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning](https://mlsanity.com/paper/2603.07918/) (2026)
- [Fourier Angle Alignment for Oriented Object Detection in Remote Sensing](https://mlsanity.com/paper/2602.23790/) (2026)
- [Statistical Characteristic-Guided Denoising for Rapid High-Resolution Transmission Electron Microscopy Imaging](https://mlsanity.com/paper/2603.18834/) (2026)
- [Boosting Zero-shot Stereo Matching using Large-scale Mixed Images Sources in the Real World](https://mlsanity.com/paper/2505.08607/) (2025)
- [3M-TI: High-Quality Mobile Thermal Imaging via Calibration-free Multi-Camera Cross-Modal Diffusion](https://mlsanity.com/paper/2511.19117/) (2025)
- [ERIENet: An Efficient RAW Image Enhancement Network under Low-Light Environment](https://mlsanity.com/paper/2512.15186/) (2025)
- [Degradation-Aware Metric Prompting for Hyperspectral Image Restoration](https://mlsanity.com/paper/2512.20251/) (2025)
- [HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding](https://mlsanity.com/paper/2608.04610/) (2026)
- [Physical Degradation Model-Guided Interferometric Hyperspectral Reconstruction with Unfolding Transformer](https://mlsanity.com/paper/2506.21880/) (2025)
- [Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations](https://mlsanity.com/paper/2602.18822/) (2026)
- [Learning Semantic-Robust Change Detection via Semantic-Invariant Self-Distillation](https://mlsanity.com/paper/2607.19000/) (2026)
- [BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion](https://mlsanity.com/paper/2607.24110/) (2026)
- [Binarized High-Efficiency RAW Video Restoration and Beyond](https://mlsanity.com/paper/2608.16756/) (2026)
- [RDANet: Relative Degradation Aware Network for Infrared Small Target Detection](https://mlsanity.com/paper/2608.20870/) (2026)
- [Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment](https://mlsanity.com/paper/2609.01060/) (2026)
- [High Dynamic Range Video Reconstruction from Single-Exposure Raw Sequences](https://mlsanity.com/paper/2609.27274/) (2026)

## Abstract (arXiv)

Hyperspectral images super-resolution aims to improve the spatial resolution, yet its performance is often limited at high-resolution ratios. The recent adoption of high-resolution reference images for super-resolution is driven by the poor spatial detail found in low-resolution HSIs, presenting it as a favorable method. However, these approaches cannot effectively utilize information from the reference image, due to the inaccuracy of alignment and its inadequate interaction between alignment and fusion modules. In this paper, we introduce a Spatial-Spectral Concordance Hyperspectral Super-Resolution (SSC-HSR) framework for unaligned reference RGB guided HSI SR to address the issues of inaccurate alignment and poor interactivity of the previous approaches. Specifically, to ensure spatial concordance, i.e., align images more accurately across resolutions and refine textures, we construct a Two-Stage Image Alignment with a synthetic generation pipeline in the image alignment module, where the fine-tuned optical flow model can produce a more accurate optical flow in the first stage and warp model can refine damaged textures in the second stage. To enhance the interaction between alignment and fusion modules and ensure spectral concordance during reconstruction, we propose a Feature Aggregation module and an Attention Fusion module. In the feature aggregation module, we introduce an Iterative Deformable Feature Aggregation block to achieve significant feature matching and texture aggregation with the fusion multi-scale results guidance, iteratively generating learnable offset. Besides, we introduce two basic spectral-wise attention blocks in the attention fusion module to model the inter-spectra interactions. Extensive experiments on three natural or remote-sensing datasets show that our method outperforms state-of-the-art approaches on both quantitative and qualitative evaluations.