# Hyperspectral vs. RGB for Pedestrian Segmentation in Urban Driving Scenes: A Comparative Study

arXiv: https://arxiv.org/abs/2508.11301 · HTML: https://mlsanity.com/paper/2508.11301/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey](https://mlsanity.com/paper/2112.08936/) (2021)
- [A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging](https://mlsanity.com/paper/2106.15944/) (2021)
- [Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models](https://mlsanity.com/paper/2410.22101/) (2024)
- [On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2411.19274/) (2024)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [CSNR and JMIM Based Spectral Band Selection for Reducing Metamerism in Urban Driving](https://mlsanity.com/paper/2508.10962/) (2025)
- [Multi-Scale Spectral Attention Module-based Hyperspectral Segmentation in Autonomous Driving Scenarios](https://mlsanity.com/paper/2506.18682/) (2025)

## Built on by (most cited first, 4 of 4)

- [Hyperspectral Sensors and Autonomous Driving: Technologies, Limitations, and Opportunities](https://mlsanity.com/paper/2508.19905/) (2025)
- [Learnable Quantum Efficiency Filters for Urban Hyperspectral Segmentation](https://mlsanity.com/paper/2603.26528/) (2026)
- [Polyhedral Unmixing: Bridging Semantic Segmentation with Hyperspectral Unmixing via Polyhedral-Cone Partitioning](https://mlsanity.com/paper/2603.29438/) (2026)
- [Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City](https://mlsanity.com/paper/2609.31074/) (2026)

## Abstract (arXiv)

Pedestrian segmentation in automotive perception systems faces critical safety challenges due to metamerism in RGB imaging, where pedestrians and backgrounds appear visually indistinguishable.. This study investigates the potential of hyperspectral imaging (HSI) for enhanced pedestrian segmentation in urban driving scenarios using the Hyperspectral City v2 (H-City) dataset. We compared standard RGB against two dimensionality-reduction approaches by converting 128-channel HSI data into three-channel representations: Principal Component Analysis (PCA) and optimal band selection using Contrast Signal-to-Noise Ratio with Joint Mutual Information Maximization (CSNR-JMIM). Three semantic segmentation models were evaluated: U-Net, DeepLabV3+, and SegFormer. CSNR-JMIM consistently outperformed RGB with an average improvements of 1.44% in Intersection over Union (IoU) and 2.18% in F1-score for pedestrian segmentation. Rider segmentation showed similar gains with 1.43% IoU and 2.25% F1-score improvements. These improved performance results from enhanced spectral discrimination of optimally selected HSI bands effectively reducing false positives. This study demonstrates robust pedestrian segmentation through optimal HSI band selection, showing significant potential for safety-critical automotive applications.