# Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction

arXiv: https://arxiv.org/abs/2109.01403 · HTML: https://mlsanity.com/paper/2109.01403/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)

## Built on by (most cited first, 7 of 7)

- [Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks](https://mlsanity.com/paper/2307.01990/) (2023)
- [Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging](https://mlsanity.com/paper/2302.10927/) (2023)
- [A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging](https://mlsanity.com/paper/2407.19282/) (2024)
- [OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations](https://mlsanity.com/paper/2411.09553/) (2024)
- [Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation](https://mlsanity.com/paper/2506.21150/) (2025)
- [Label tree semantic losses for rich multi-class medical image segmentation](https://mlsanity.com/paper/2507.15777/) (2025)
- [Perspective-Equivariant Fine-tuning for Multispectral Demosaicing without Ground Truth](https://mlsanity.com/paper/2603.01332/) (2026)

## Abstract (arXiv)

Hyperspectral imaging is one of the most promising techniques for intraoperative tissue characterisation. Snapshot mosaic cameras, which can capture hyperspectral data in a single exposure, have the potential to make a real-time hyperspectral imaging system for surgical decision-making possible. However, optimal exploitation of the captured data requires solving an ill-posed demosaicking problem and applying additional spectral corrections to recover spatial and spectral information of the image. In this work, we propose a deep learning-based image demosaicking algorithm for snapshot hyperspectral images using supervised learning methods. Due to the lack of publicly available medical images acquired with snapshot mosaic cameras, a synthetic image generation approach is proposed to simulate snapshot images from existing medical image datasets captured by high-resolution, but slow, hyperspectral imaging devices. Image reconstruction is achieved using convolutional neural networks for hyperspectral image super-resolution, followed by cross-talk and leakage correction using a sensor-specific calibration matrix. The resulting demosaicked images are evaluated both quantitatively and qualitatively, showing clear improvements in image quality compared to a baseline demosaicking method using linear interpolation. Moreover, the fast processing time of~45,ms of our algorithm to obtain super-resolved RGB or oxygenation saturation maps per image frame for a state-of-the-art snapshot mosaic camera demonstrates the potential for its seamless integration into real-time surgical hyperspectral imaging applications.