# Risky Action Recognition in Lane Change Video Clips using Deep Spatiotemporal Networks with Segmentation Mask Transfer

arXiv: https://arxiv.org/abs/1906.02859 · HTML: https://mlsanity.com/paper/1906.02859/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [RS2G: Data-Driven Scene-Graph Extraction and Embedding for Robust Autonomous Perception and Scenario Understanding](https://mlsanity.com/paper/2304.08600/) (2023)
- [roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphs](https://mlsanity.com/paper/2109.01183/) (2021)
- [DROID: Driver-centric Risk Object Identification](https://mlsanity.com/paper/2106.13201/) (2021)
- [What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studies](https://mlsanity.com/paper/2507.06513/) (2025)
- [Monocular Vision-based Prediction of Cut-in Maneuvers with LSTM Networks](https://mlsanity.com/paper/2203.10707/) (2022)
- [Integrating Legal and Logical Specifications in Perception, Prediction, and Planning for Automated Driving: A Survey of Methods](https://mlsanity.com/paper/2510.25386/) (2025)

## Abstract (arXiv)

Advanced driver assistance and automated driving systems rely on risk estimation modules to predict and avoid dangerous situations. Current methods use expensive sensor setups and complex processing pipeline, limiting their availability and robustness. To address these issues, we introduce a novel deep learning based action recognition framework for classifying dangerous lane change behavior in short video clips captured by a monocular camera. We designed a deep spatiotemporal classification network that uses pre-trained state-of-the-art instance segmentation network Mask R-CNN as its spatial feature extractor for this task. The Long-Short Term Memory (LSTM) and shallower final classification layers of the proposed method were trained on a semi-naturalistic lane change dataset with annotated risk labels. A comprehensive comparison of state-of-the-art feature extractors was carried out to find the best network layout and training strategy. The best result, with a 0.937 AUC score, was obtained with the proposed network. Our code and trained models are available open-source.