# HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios

arXiv: https://arxiv.org/abs/2409.11205 · HTML: https://mlsanity.com/paper/2409.11205/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017)
- [The Devil is in the Decoder: Classification, Regression and GANs](https://mlsanity.com/paper/1707.05847/) (2017)

## Built on by (most cited first, 12 of 12)

- [Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2509.20107/) (2025)
- [Hyperspectral Sensors and Autonomous Driving: Technologies, Limitations, and Opportunities](https://mlsanity.com/paper/2508.19905/) (2025)
- [Multi-Scale Spectral Attention Module-based Hyperspectral Segmentation in Autonomous Driving Scenarios](https://mlsanity.com/paper/2506.18682/) (2025)
- [CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis](https://mlsanity.com/paper/2504.19223/) (2025)
- [HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone](https://mlsanity.com/paper/2605.17286/) (2026)
- [SLTNet: Efficient Event-based Semantic Segmentation with Spike-driven Lightweight Transformer-based Networks](https://mlsanity.com/paper/2412.12843/) (2024)
- [Optimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach](https://mlsanity.com/paper/2507.16556/) (2025)
- [Data-Efficient Spectral Classification of Hyperspectral Data Using MiniROCKET and HDC-MiniROCKET](https://mlsanity.com/paper/2509.13809/) (2025)
- [Cuvis.Ai: An Open-Source, Low-Code Software Ecosystem for Hyperspectral Processing and Classification](https://mlsanity.com/paper/2411.11324/) (2024)
- [Exploring Spatiotemporal Feature Propagation for Video-Level Compressive Spectral Reconstruction: Dataset, Model and Benchmark](https://mlsanity.com/paper/2603.00611/) (2026)
- [Cross-Domain Transfer of Hyperspectral Foundation Models](https://mlsanity.com/paper/2604.26478/) (2026)
- [Perception Beyond the Visible: Incompleteness-Aware Scene Understanding for Robot Autonomy](https://mlsanity.com/paper/2610.18670/) (2026)

## Abstract (arXiv)

Semantic segmentation is an essential step for many vision applications in order to understand a scene and the objects within. Recent progress in hyperspectral imaging technology enables the application in driving scenarios and the hope is that the devices perceptive abilities provide an advantage over RGB-cameras. Even though some datasets exist, there is no standard benchmark available to systematically measure progress on this task and evaluate the benefit of hyperspectral data. In this paper, we work towards closing this gap by providing the HyperSpectral Semantic Segmentation benchmark (HS3-Bench). It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols. We use the benchmark to derive two strong baseline models that surpass the previous state-of-the-art performances with and without pre-training on the individual datasets. Further, our results indicate that the existing learning-based methods benefit more from leveraging additional RGB training data than from leveraging the additional hyperspectral channels. This poses important questions for future research on hyperspectral imaging for semantic segmentation in driving scenarios. Code to run the benchmark and the strong baseline approaches are available under https://github.com/nickstheisen/hyperseg.