# Registration-Free Hyperspectral Reconstruction from RGB via a Permutation-Invariant Gram-Matrix Principle

arXiv: https://arxiv.org/abs/2608.14994 · HTML: https://mlsanity.com/paper/2608.14994/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 15 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration](https://mlsanity.com/paper/1709.00192/) (2017)
- [Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion](https://mlsanity.com/paper/2008.03426/) (2020)
- [Joint and Progressive Subspace Analysis (JPSA) with Spatial-Spectral Manifold Alignment for Semi-Supervised Hyperspectral Dimensionality Reduction](https://mlsanity.com/paper/2009.10003/) (2020)
- [Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net](https://mlsanity.com/paper/1904.12175/) (2019)
- [Towards Spectral Estimation from a Single RGB Image in the Wild](https://mlsanity.com/paper/1812.00805/) (2018)
- [Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning](https://mlsanity.com/paper/2603.07918/) (2026)
- [Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability](https://mlsanity.com/paper/2603.21510/) (2026)

## Abstract (arXiv)

Reconstructing a spatially and spectrally high-resolution hyperspectral image (HR-HSI) from a low-resolution HSI (LR-HSI) and a high-resolution RGB image (HR-RGB) usually assumes precise registration and a known camera response function (CRF). Both assumptions are difficult to satisfy with different sensors. We remove both through a permutation-invariant supervision principle: the Gram matrix of an unmixed abundance map depends on shared material composition but not on pixel ordering. Matching abundance Gram matrices therefore allows RGB-to-HSI mapping to be learned without spatial correspondence and without a predefined CRF. Under a full random permutation of HR-RGB pixels, a state-of-the-art fusion method collapses, whereas our reconstruction is unchanged after inverse reindexing for evaluation. Building on this principle, a residual spectral super-resolution function maps HR-RGB directly to HR-HSI without registration, known CRF, or paired supervision. Across indoor, natural-scene, and remote-sensing benchmarks, the method achieves accuracy comparable to approaches that require these assumptions while remaining robust when they are violated. Loss ablations further show that reconstruction accuracy is largely insensitive to the specific discrepancy used to match the Gram matrices, indicating that performance arises primarily from the permutation-invariant principle rather than loss tuning.