# Video-rate multispectral imaging in laparoscopic surgery: First-in-human application

arXiv: https://arxiv.org/abs/2105.13901 · HTML: https://mlsanity.com/paper/2105.13901/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 84 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans](https://mlsanity.com/paper/2008.06388/) (2020)
- [Discriminative Correlation Filter with Channel and Spatial Reliability](https://mlsanity.com/paper/1611.08461/) (2016)
- [Neural Network Generalization: The impact of camera parameters](https://mlsanity.com/paper/1912.03604/) (2019)
- [Medical Imaging and Machine Learning](https://mlsanity.com/paper/2103.01938/) (2021)
- [Estimation of blood oxygenation with learned spectral decoloring for quantitative photoacoustic imaging (LSD-qPAI)](https://mlsanity.com/paper/1902.05839/) (2019)
- [Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks](https://mlsanity.com/paper/1903.03441/) (2019)
- [Estimation of Tissue Oxygen Saturation from RGB images and Sparse Hyperspectral Signals based on Conditional Generative Adversarial Network](https://mlsanity.com/paper/1905.00391/) (2019)
- [Out of distribution detection for intra-operative functional imaging](https://mlsanity.com/paper/1911.01877/) (2019)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models](https://mlsanity.com/paper/2404.14955/) (2024)
- [OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations](https://mlsanity.com/paper/2411.09553/) (2024)
- [CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis](https://mlsanity.com/paper/2504.19223/) (2025)
- [Deep intra-operative illumination calibration of hyperspectral cameras](https://mlsanity.com/paper/2409.07094/) (2024)
- [Xeno-learning: knowledge transfer across species in deep learning-based spectral image analysis](https://mlsanity.com/paper/2410.19789/) (2024)
- [Clinical Validation of Deep Learning for Real-Time Tissue Oxygenation Estimation Using Spectral Imaging](https://mlsanity.com/paper/2505.18010/) (2025)

## Abstract (arXiv)

Multispectral and hyperspectral imaging (MSI/HSI) can provide clinically relevant information on morphological and functional tissue properties. Application in the operating room (OR), however, has so far been limited by complex hardware setups and slow acquisition times. To overcome these limitations, we propose a novel imaging system for video-rate spectral imaging in the clinical workflow. The system integrates a small snapshot multispectral camera with a standard laparoscope and a clinically commonly used light source, enabling the recording of multispectral images with a spectral dimension of 16 at a frame rate of 25 Hz. An ongoing in patient study shows that multispectral recordings from this system can help detect perfusion changes in partial nephrectomy surgery, thus opening the doors to a wide range of clinical applications.