# Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment

arXiv: https://arxiv.org/abs/2609.01060 · HTML: https://mlsanity.com/paper/2609.01060/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance](https://mlsanity.com/paper/2505.02109/) (2025)
- [Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists](https://mlsanity.com/paper/1912.00137/) (2019)
- [Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance](https://mlsanity.com/paper/2302.06298/) (2023)
- [Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives](https://mlsanity.com/paper/2407.01355/) (2024)
- [Hyper-Skin: A Hyperspectral Dataset for Reconstructing Facial Skin-Spectra from RGB Images](https://mlsanity.com/paper/2310.17911/) (2023)
- [RGB-Guided Resolution Enhancement of IR Images](https://mlsanity.com/paper/2309.05996/) (2023)
- [Spectro-spatial hyperspectral image reconstruction from interferometric acquisitions](https://mlsanity.com/paper/2310.01898/) (2023)
- [Multiple-beam Interference Spectroscopy: Instrument Analysis and Spectrum Reconstruction](https://mlsanity.com/paper/2410.21586/) (2024)
- [Leveraging pretrained RGB denoisers for hyperspectral image restoration](https://mlsanity.com/paper/2605.24769/) (2026)

## Abstract (arXiv)

Compact snapshot hyperspectral cameras provide rich instantaneous spectral measurements for ground-level machine vision, but at lower spatial resolution than standard RGB cameras. RGB-guided hyperspectral super-resolution (HSR) addresses this limitation by transferring spatial detail from a high-resolution RGB guide to a low-resolution hyperspectral image (HSI). These dual-camera systems are typically in a horizontal rig geometry, requiring cross-camera image alignment due to different fields of view. However, residual misregistration can inject spurious high-frequency details. Existing learned unaligned-fusion methods are usually trained for a fixed spectral support and spatial scale factors and can be computationally demanding, limiting their flexibility across sensors. We propose a lightweight and interpretable RGB-guided HSR framework combining cross-modal flow alignment with model-based Gram-Schmidt orthogonalization fusion. The method first warps the RGB guide onto the HSI grid, then estimates an energy-based confidence weight map by measuring local alignment reliability. This map is then used both in a weighted least-squares spectral regression and in a gated fusion between the super-resolved estimate and an HSI-preserving estimate. Unlike existing learned methods, the proposed framework has a low computational footprint and supports VIS-NIR spectral supports and scale factors without retraining. Experiments on the Real benchmark show that the proposed method improves reconstruction accuracy over learned fusion baselines while remaining substantially faster. On a 34-frame sequence acquired with our real RGB-HSI dual-camera setup, a reduced-resolution quantitative evaluation validates the method under genuine cross-sensor radiometric, noise, and geometric differences, while native-resolution qualitative results demonstrate deployment on the full 51-band VIS-NIR acquisition.