# A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

arXiv: https://arxiv.org/abs/2106.10319 · HTML: https://mlsanity.com/paper/2106.10319/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning](https://mlsanity.com/paper/1811.02307/) (2018)
- [Joint Monocular 3D Vehicle Detection and Tracking](https://mlsanity.com/paper/1811.10742/) (2018)
- [A CNN-RNN Architecture for Multi-Label Weather Recognition](https://mlsanity.com/paper/1904.10709/) (2019)

## Built on by (most cited first, 3 of 3)

- [Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happen](https://mlsanity.com/paper/2505.21743/) (2025)
- [Crash Report Data Analysis for Creating Scenario-Wise, Spatio-Temporal Attention Guidance to Support Computer Vision-based Perception of Fatal Crash Risks](https://mlsanity.com/paper/2109.02710/) (2021)
- [The Context of Crash Occurrence: A Complexity-Infused Approach Integrating Semantic, Contextual, and Kinematic Features](https://mlsanity.com/paper/2411.17886/) (2024)

## Abstract (arXiv)

To assist human drivers and autonomous vehicles in assessing crash risks, driving scene analysis using dash cameras on vehicles and deep learning algorithms is of paramount importance. Although these technologies are increasingly available, driving scene analysis for this purpose still remains a challenge. This is mainly due to the lack of annotated large image datasets for analyzing crash risk indicators and crash likelihood, and the lack of an effective method to extract lots of required information from complex driving scenes. To fill the gap, this paper develops a scene analysis system. The Multi-Net of the system includes two multi-task neural networks that perform scene classification to provide four labels for each scene. The DeepLab v3 and YOLO v3 are combined by the system to detect and locate risky pedestrians and the nearest vehicles. All identified information can provide the situational awareness to autonomous vehicles or human drivers for identifying crash risks from the surrounding traffic. To address the scarcity of annotated image datasets for studying traffic crashes, two completely new datasets have been developed by this paper and made available to the public, which were proved to be effective in training the proposed deep neural networks. The paper further evaluates the performance of the Multi-Net and the efficiency of the developed system. Comprehensive scene analysis is further illustrated with representative examples. Results demonstrate the effectiveness of the developed system and datasets for driving scene analysis, and their supportiveness for crash risk assessment and crash prevention.