# mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

arXiv: https://arxiv.org/abs/2303.16205 · HTML: https://mlsanity.com/paper/2303.16205/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 122 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [Discovering physical concepts with neural networks](https://mlsanity.com/paper/1807.10300/) (2018)
- [DiffuserCam: Lensless Single-exposure 3D Imaging](https://mlsanity.com/paper/1710.02134/) (2017)
- [NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image](https://mlsanity.com/paper/2005.03412/) (2020)
- [Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation](https://mlsanity.com/paper/1502.03121/) (2015)
- [Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach](https://mlsanity.com/paper/1804.05307/) (2018)
- [Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array](https://mlsanity.com/paper/2006.08565/) (2020)
- [A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging](https://mlsanity.com/paper/2106.15944/) (2021)
- [Learned Spectral Super-Resolution](https://mlsanity.com/paper/1703.09470/) (2017)
- [Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN](https://mlsanity.com/paper/1806.03575/) (2018)
- [Towards Spectral Estimation from a Single RGB Image in the Wild](https://mlsanity.com/paper/1812.00805/) (2018)
- [Compressive Spectral Image Reconstruction using Deep Prior and Low-Rank Tensor Representation](https://mlsanity.com/paper/2101.07424/) (2021)
- [Dynamic brain spectrum acquired by a real-time ultra-spectral imaging chip with reconfigurable metasurfaces](https://mlsanity.com/paper/2005.02689/) (2020)
- [Neural network-based on-chip spectroscopy using a scalable plasmonic encoder](https://mlsanity.com/paper/2012.00878/) (2020)
- [Dual camera snapshot hyperspectral imaging system via physics informed learning](https://mlsanity.com/paper/2109.02643/) (2021)

## Built on by (most cited first, 1 of 1)

- [Limitations of Data-Driven Spectral Reconstruction -- An Optics-Aware Analysis](https://mlsanity.com/paper/2401.03835/) (2024)

## Abstract (arXiv)

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from an RGB image. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep-learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral tradeoff, offering simple hardware requirements and potential applications of various machine-learning techniques.