# An Attention-guided Multistream Feature Fusion Network for Localization of Risky Objects in Driving Videos

arXiv: https://arxiv.org/abs/2209.07922 · HTML: https://mlsanity.com/paper/2209.07922/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learning](https://mlsanity.com/paper/2008.00334/) (2020)
- [Abnormal Event Detection in Videos using Spatiotemporal Autoencoder](https://mlsanity.com/paper/1701.01546/) (2017)
- [DRIVE: Deep Reinforced Accident Anticipation with Visual Explanation](https://mlsanity.com/paper/2107.10189/) (2021)
- [Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systems](https://mlsanity.com/paper/1809.07408/) (2018)
- [A Dynamic Spatial-temporal Attention Network for Early Anticipation of Traffic Accidents](https://mlsanity.com/paper/2106.10197/) (2021)
- [Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference](https://mlsanity.com/paper/2003.02425/) (2020)
- [Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization](https://mlsanity.com/paper/1705.06560/) (2017)
- [Goal-oriented Object Importance Estimation in On-road Driving Videos](https://mlsanity.com/paper/1905.02848/) (2019)
- [Crash Report Data Analysis for Creating Scenario-Wise, Spatio-Temporal Attention Guidance to Support Computer Vision-based Perception of Fatal Crash Risks](https://mlsanity.com/paper/2109.02710/) (2021)

## Built on by (most cited first, 4 of 4)

- [Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark](https://mlsanity.com/paper/2212.09381/) (2022)
- [DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding](https://mlsanity.com/paper/2311.06497/) (2023)
- [CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios](https://mlsanity.com/paper/2608.19380/) (2026)
- [A Two-Stage Framework for Ego-Centric Key Object Identification via Object State Prediction](https://mlsanity.com/paper/2609.07125/) (2026)

## Abstract (arXiv)

Detecting dangerous traffic agents in videos captured by vehicle-mounted dashboard cameras (dashcams) is essential to facilitate safe navigation in a complex environment. Accident-related videos are just a minor portion of the driving video big data, and the transient pre-accident processes are highly dynamic and complex. Besides, risky and non-risky traffic agents can be similar in their appearance. These make risky object localization in the driving video particularly challenging. To this end, this paper proposes an attention-guided multistream feature fusion network (AM-Net) to localize dangerous traffic agents from dashcam videos. Two Gated Recurrent Unit (GRU) networks use object bounding box and optical flow features extracted from consecutive video frames to capture spatio-temporal cues for distinguishing dangerous traffic agents. An attention module coupled with the GRUs learns to attend to the traffic agents relevant to an accident. Fusing the two streams of features, AM-Net predicts the riskiness scores of traffic agents in the video. In supporting this study, the paper also introduces a benchmark dataset called Risky Object Localization (ROL). The dataset contains spatial, temporal, and categorical annotations with the accident, object, and scene-level attributes. The proposed AM-Net achieves a promising performance of 85.73% AUC on the ROL dataset. Meanwhile, the AM-Net outperforms current state-of-the-art for video anomaly detection by 6.3% AUC on the DoTA dataset. A thorough ablation study further reveals AM-Net's merits by evaluating the contributions of its different components.