# Multispectral Stereo-Image Fusion for 3D Hyperspectral Scene Reconstruction

arXiv: https://arxiv.org/abs/2401.09428 · HTML: https://mlsanity.com/paper/2401.09428/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 61 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 5 of 5)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image](https://mlsanity.com/paper/2005.03412/) (2020)
- [High Resolution Multi-Scale RAFT (Robust Vision Challenge 2022)](https://mlsanity.com/paper/2210.16900/) (2022)
- [Hyperspectral Demosaicing of Snapshot Camera Images Using Deep Learning](https://mlsanity.com/paper/2211.15435/) (2022)

## Built on by (most cited first, 1 of 1)

- [Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging](https://mlsanity.com/paper/2608.28341/) (2026)

## Abstract (arXiv)

Spectral imaging enables the analysis of optical material properties that are invisible to the human eye. Different spectral capturing setups, e.g., based on filter-wheel, push-broom, line-scanning, or mosaic cameras, have been introduced in the last years to support a wide range of applications in agriculture, medicine, and industrial surveillance. However, these systems often suffer from different disadvantages, such as lack of real-time capability, limited spectral coverage or low spatial resolution. To address these drawbacks, we present a novel approach combining two calibrated multispectral real-time capable snapshot cameras, covering different spectral ranges, into a stereo-system. Therefore, a hyperspectral data-cube can be continuously captured. The combined use of different multispectral snapshot cameras enables both 3D reconstruction and spectral analysis. Both captured images are demosaicked avoiding spatial resolution loss. We fuse the spectral data from one camera into the other to receive a spatially and spectrally high resolution video stream. Experiments demonstrate the feasibility of this approach and the system is investigated with regard to its applicability for surgical assistance monitoring.