# Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks

arXiv: https://arxiv.org/abs/2311.11782 · HTML: https://mlsanity.com/paper/2311.11782/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 74 references that are in the map.
- First posted on arXiv 20 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Deep Learning for Hyperspectral Image Classification: An Overview](https://mlsanity.com/paper/1910.12861/) (2019)
- [The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation](https://mlsanity.com/paper/1611.09326/) (2016)
- [Graph Convolutional Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2008.02457/) (2020)
- [Towards a Guideline for Evaluation Metrics in Medical Image Segmentation](https://mlsanity.com/paper/2202.05273/) (2022)
- [Multi-scale Dynamic Graph Convolutional Network for Hyperspectral Image Classification](https://mlsanity.com/paper/1905.06133/) (2019)
- [A joint 3D UNet-Graph Neural Network-based method for Airway Segmentation from chest CTs](https://mlsanity.com/paper/1908.08588/) (2019)
- [Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis](https://mlsanity.com/paper/2203.10977/) (2022)
- [SpecTr: Spectral Transformer for Hyperspectral Pathology Image Segmentation](https://mlsanity.com/paper/2103.03604/) (2021)
- [Hyperspectral Image Classification With Context-Aware Dynamic Graph Convolutional Network](https://mlsanity.com/paper/1909.11953/) (2019)
- [Trends in deep learning for medical hyperspectral image analysis](https://mlsanity.com/paper/2011.13974/) (2020)
- [Spectral Pyramid Graph Attention Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2001.07108/) (2020)
- [Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy](https://mlsanity.com/paper/1706.07002/) (2017)
- [A Joint Graph and Image Convolution Network for Automatic Brain Tumor Segmentation](https://mlsanity.com/paper/2109.05580/) (2021)
- [Multi-class Brain Tumor Segmentation using Graph Attention Network](https://mlsanity.com/paper/2302.05598/) (2023)
- [Intra-operative Brain Tumor Detection with Deep Learning-Optimized Hyperspectral Imaging](https://mlsanity.com/paper/2302.02884/) (2023)

## Built on by (most cited first, 1 of 1)

- [Morphological Decoupling-Based Skeletal Classification for Clinical Assessment of Malocclusion](https://mlsanity.com/paper/2609.09801/) (2026)

## Abstract (arXiv)

Segmenting the boundary between tumor and healthy tissue during surgical cancer resection poses a significant challenge. In recent years, Hyperspectral Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising solution. However, due to the extensive information contained within the spectral domain, most ML approaches primarily classify individual HSI (super-)pixels, or tiles, without taking into account their spatial context. In this paper, we propose an improved methodology that leverages the spatial context of tiles for more robust and smoother segmentation. To address the irregular shapes of tiles, we utilize Graph Neural Networks (GNNs) to propagate context information across neighboring regions. The features for each tile within the graph are extracted using a Convolutional Neural Network (CNN), which is trained simultaneously with the subsequent GNN. Moreover, we incorporate local image quality metrics into the loss function to enhance the training procedure's robustness against low-quality regions in the training images. We demonstrate the superiority of our proposed method using a clinical ex vivo dataset consisting of 51 HSI images from 30 patients. Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients. Furthermore, we show that our carefully designed loss function, accounting for local image quality, results in additional improvements. Our findings demonstrate that context-aware GNN algorithms can robustly find tumor demarcations on HSI images, ultimately contributing to better surgery success and patient outcome.