# RGB to Hyperspectral: Spectral Reconstruction for Enhanced Surgical Imaging

arXiv: https://arxiv.org/abs/2410.13570 · HTML: https://mlsanity.com/paper/2410.13570/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging](https://mlsanity.com/paper/2106.15944/) (2021)
- [Learned Spectral Super-Resolution](https://mlsanity.com/paper/1703.09470/) (2017)
- [Advancing dermatological diagnosis: Development of a hyperspectral dermatoscope for enhanced skin imaging](https://mlsanity.com/paper/2403.00612/) (2024)

## Built on by (most cited first, 2 of 2)

- [SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation](https://mlsanity.com/paper/2507.23673/) (2025)
- [AI-Enhanced Hyperspectral Imaging for Intraoperative Detection of Low-Grade Glioma: A Clinical Study](https://mlsanity.com/paper/2610.17944/) (2026)

## Abstract (arXiv)

This study investigates the reconstruction of hyperspectral signatures from RGB data to enhance surgical imaging, utilizing the publicly available HeiPorSPECTRAL dataset from porcine surgery and an in-house neurosurgery dataset. Various architectures based on convolutional neural networks (CNNs) and transformer models are evaluated using comprehensive metrics. Transformer models exhibit superior performance in terms of RMSE, SAM, PSNR and SSIM by effectively integrating spatial information to predict accurate spectral profiles, encompassing both visible and extended spectral ranges. Qualitative assessments demonstrate the capability to predict spectral profiles critical for informed surgical decision-making during procedures. Challenges associated with capturing both the visible and extended hyperspectral ranges are highlighted using the MAE, emphasizing the complexities involved. The findings open up the new research direction of hyperspectral reconstruction for surgical applications and clinical use cases in real-time surgical environments.