# Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models

arXiv: https://arxiv.org/abs/2410.22101 · HTML: https://mlsanity.com/paper/2410.22101/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 10 of 10)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Weakly-supervised Semantic Segmentation in Cityscape via Hyperspectral Image](https://mlsanity.com/paper/2012.10122/) (2020)
- [Onboard Processing of Hyperspectral Imagery: Deep Learning Advancements, Methodologies, Challenges, and Emerging Trends](https://mlsanity.com/paper/2404.06526/) (2024)

## Built on by (most cited first, 8 of 8)

- [Hyperspectral Sensors and Autonomous Driving: Technologies, Limitations, and Opportunities](https://mlsanity.com/paper/2508.19905/) (2025)
- [CSNR and JMIM Based Spectral Band Selection for Reducing Metamerism in Urban Driving](https://mlsanity.com/paper/2508.10962/) (2025)
- [Hyperspectral vs. RGB for Pedestrian Segmentation in Urban Driving Scenes: A Comparative Study](https://mlsanity.com/paper/2508.11301/) (2025)
- [Multi-Scale Spectral Attention Module-based Hyperspectral Segmentation in Autonomous Driving Scenarios](https://mlsanity.com/paper/2506.18682/) (2025)
- [Learnable Quantum Efficiency Filters for Urban Hyperspectral Segmentation](https://mlsanity.com/paper/2603.26528/) (2026)
- [Challenges in Hyperspectral Imaging for Autonomous Driving: The HSI-Drive Case](https://mlsanity.com/paper/2603.25510/) (2026)
- [HSI-Road Relabeled: Surface-Aware Road-Scene Segmentation](https://mlsanity.com/paper/2609.12151/) (2026)
- [Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City](https://mlsanity.com/paper/2609.31074/) (2026)

## Abstract (arXiv)

Hyperspectral Imaging (HSI) is known for its advantages over traditional RGB imaging in remote sensing, agriculture, and medicine. Recently, it has gained attention for enhancing Advanced Driving Assistance Systems (ADAS) perception. Several HSI datasets such as HyKo, HSI-Drive, HSI-Road, and Hyperspectral City have been made available. However, a comprehensive evaluation of semantic segmentation models (SSM) using these datasets is lacking. To address this gap, we evaluated the available annotated HSI datasets on four deep learning-based baseline SSMs: DeepLab v3+, HRNet, PSPNet, and U-Net, along with its two variants: Coordinate Attention (UNet-CA) and Convolutional Block-Attention Module (UNet-CBAM). The original model architectures were adapted to handle the varying spatial and spectral dimensions of the datasets. These baseline SSMs were trained using a class-weighted loss function for individual HSI datasets and evaluated using mean-based metrics such as intersection over union (IoU), recall, precision, F1 score, specificity, and accuracy. Our results indicate that UNet-CBAM, which extracts channel-wise features, outperforms other SSMs and shows potential to leverage spectral information for enhanced semantic segmentation. This study establishes a baseline SSM benchmark on available annotated datasets for future evaluation of HSI-based ADAS perception. However, limitations of current HSI datasets, such as limited dataset size, high class imbalance, and lack of fine-grained annotations, remain significant constraints for developing robust SSMs for ADAS applications.