# Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild

arXiv: https://arxiv.org/abs/2108.06659 · HTML: https://mlsanity.com/paper/2108.06659/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 54 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Weakly-Supervised Semantic Segmentation via Sub-category Exploration](https://mlsanity.com/paper/2008.01183/) (2020)
- [Cross-View Image Synthesis using Conditional GANs](https://mlsanity.com/paper/1803.03396/) (2018)
- [Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms](https://mlsanity.com/paper/1812.04179/) (2018)
- [NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image](https://mlsanity.com/paper/2005.03412/) (2020)
- [AdaptiveWeighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images](https://mlsanity.com/paper/2005.09305/) (2020)
- [Learned Spectral Super-Resolution](https://mlsanity.com/paper/1703.09470/) (2017)
- [Pixel-aware Deep Function-mixture Network for Spectral Super-Resolution](https://mlsanity.com/paper/1903.10501/) (2019)
- [Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning](https://mlsanity.com/paper/2006.10300/) (2020)
- [Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB](https://mlsanity.com/paper/1709.00265/) (2017)
- [Towards Spectral Estimation from a Single RGB Image in the Wild](https://mlsanity.com/paper/1812.00805/) (2018)

## Built on by (most cited first, 6 of 6)

- [Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers](https://mlsanity.com/paper/2307.04129/) (2023)
- [Deep Posterior Distribution-based Embedding for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2205.14887/) (2022)
- [GMSR:Gradient-Guided Mamba for Spectral Reconstruction from RGB Images](https://mlsanity.com/paper/2405.07777/) (2024)
- [Continuous Spectral Reconstruction from RGB Images via Implicit Neural Representation](https://mlsanity.com/paper/2112.13003/) (2021)
- [From Image- to Pixel-level: Label-efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2503.06852/) (2025)
- [NukesFormers: Unpaired Hyperspectral Image Generation with Non-Uniform Domain Alignment](https://mlsanity.com/paper/2503.07004/) (2025)

## Abstract (arXiv)

This paper investigates the problem of reconstructing hyperspectral (HS) images from single RGB images captured by commercial cameras, without using paired HS and RGB images during training. To tackle this challenge, we propose a new lightweight and end-to-end learning-based framework. Specifically, on the basis of the intrinsic imaging degradation model of RGB images from HS images, we progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective unsupervised camera spectral response function estimation. To enable the learning without paired ground-truth HS images as supervision, we adopt the adversarial learning manner and boost it with a simple yet effective L1 gradient clipping scheme. Besides, we embed the semantic information of input RGB images to locally regularize the unsupervised learning, which is expected to promote pixels with identical semantics to have consistent spectral signatures. In addition to conducting quantitative experiments over two widely-used datasets for HS image reconstruction from synthetic RGB images, we also evaluate our method by applying recovered HS images from real RGB images to HS-based visual tracking. Extensive results show that our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings. The source code is public available at https://github.com/zbzhzhy/Unsupervised-Spectral-Reconstruction.