# Optimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach arXiv: https://arxiv.org/abs/2507.16556 ยท HTML: https://mlsanity.com/paper/2507.16556/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 63 references that are in the map. - First posted on arXiv 22 Jul 2025. ## Builds on (most cited first, 29 of 29) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014) - [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016) - [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018) - [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018) - [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018) - [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016) - [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017) - [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018) - [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019) - [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022) - [Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning](https://mlsanity.com/paper/2208.11580/) (2022) - [What's Hidden in a Randomly Weighted Neural Network?](https://mlsanity.com/paper/1911.13299/) (2019) - [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://mlsanity.com/paper/1905.01067/) (2019) - [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023) - [Proving the Lottery Ticket Hypothesis: Pruning is All You Need](https://mlsanity.com/paper/2002.00585/) (2020) - [A Fast Post-Training Pruning Framework for Transformers](https://mlsanity.com/paper/2204.09656/) (2022) - [Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions](https://mlsanity.com/paper/1711.08141/) (2017) - [Strong but simple: A Baseline for Domain Generalized Dense Perception by CLIP-based Transfer Learning](https://mlsanity.com/paper/2312.02021/) (2023) - [Dynamically pruning segformer for efficient semantic segmentation](https://mlsanity.com/paper/2111.09499/) (2021) - [HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios](https://mlsanity.com/paper/2409.11205/) (2024) - [HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2411.17530/) (2024) - [Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning](https://mlsanity.com/paper/2406.01820/) (2024) - [Evaluating Single Event Upsets in Deep Neural Networks for Semantic Segmentation: an embedded system perspective](https://mlsanity.com/paper/2412.03630/) (2024) - [Hyperspectral City V1.0 Dataset and Benchmark](https://mlsanity.com/paper/1907.10270/) (2019) - [On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2411.19274/) (2024) - [Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environments](https://mlsanity.com/paper/2308.08058/) (2023) - [Auto-Compressing Subset Pruning for Semantic Image Segmentation](https://mlsanity.com/paper/2201.11103/) (2022) - [Rapid Deployment of Domain-specific Hyperspectral Image Processors with Application to Autonomous Driving](https://mlsanity.com/paper/2411.17543/) (2024) ## Built on by (most cited first, 1 of 1) - [Hyperspectral Sensors and Autonomous Driving: Technologies, Limitations, and Opportunities](https://mlsanity.com/paper/2508.19905/) (2025) ## Abstract (arXiv) The use of HSI for autonomous navigation is a promising research field aimed at improving the accuracy and robustness of detection, tracking, and scene understanding systems based on vision sensors. Combining advanced computer algorithms, such as DNNs, with small-size snapshot HSI cameras enhances the reliability of these systems. HSI overcomes intrinsic limitations of greyscale and RGB imaging in depicting physical properties of targets, particularly regarding spectral reflectance and metamerism. Despite promising results in HSI-based vision developments, safety-critical systems like ADS demand strict constraints on latency, resource consumption, and security, motivating the shift of ML workloads to edge platforms. This involves a thorough software/hardware co-design scheme to distribute and optimize the tasks efficiently among the limited resources of computing platforms. With respect to inference, the over-parameterized nature of DNNs poses significant computational challenges for real-time on-the-edge deployment. In addition, the intensive data preprocessing required by HSI, which is frequently overlooked, must be carefully managed in terms of memory arrangement and inter-task communication to enable an efficient integrated pipeline design on a SoC. This work presents a set of optimization techniques for the practical co-design of a DNN-based HSI segmentation processor deployed on a FPGA-based SoC targeted at ADS, including key optimizations such as functional software/hardware task distribution, hardware-aware preprocessing, ML model compression, and a complete pipelined deployment. Applied compression techniques significantly reduce the complexity of the designed DNN to 24.34% of the original operations and to 1.02% of the original number of parameters, achieving a 2.86x speed-up in the inference task without noticeable degradation of the segmentation accuracy.