# Video action recognition for lane-change classification and prediction of surrounding vehicles

arXiv: https://arxiv.org/abs/2101.05043 · HTML: https://mlsanity.com/paper/2101.05043/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 51 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Convolutional Social Pooling for Vehicle Trajectory Prediction](https://mlsanity.com/paper/1805.06771/) (2018)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs](https://mlsanity.com/paper/1805.05499/) (2018)
- [Two-Stream Networks for Lane-Change Prediction of Surrounding Vehicles](https://mlsanity.com/paper/2008.10869/) (2020)
- [Predicting Future Lane Changes of Other Highway Vehicles using RNN-based Deep Models](https://mlsanity.com/paper/1801.04340/) (2018)
- [The PREVENTION Challenge: How Good Are Humans Predicting Lane Changes?](https://mlsanity.com/paper/2009.05331/) (2020)

## Built on by (most cited first, 14 of 14)

- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges](https://mlsanity.com/paper/2308.10407/) (2023)
- [Vision-based Vehicle Speed Estimation: A Survey](https://mlsanity.com/paper/2101.06159/) (2021)
- [Predicting Vehicles Trajectories in Urban Scenarios with Transformer Networks and Augmented Information](https://mlsanity.com/paper/2106.00559/) (2021)
- [Towards trustworthy multi-modal motion prediction: Holistic evaluation and interpretability of outputs](https://mlsanity.com/paper/2210.16144/) (2022)
- [ADD: An Automatic Desensitization Fisheye Dataset for Autonomous Driving](https://mlsanity.com/paper/2308.07590/) (2023)
- [Data-driven vehicle speed detection from synthetic driving simulator images](https://mlsanity.com/paper/2104.09903/) (2021)
- [Towards view-invariant vehicle speed detection from driving simulator images](https://mlsanity.com/paper/2206.00343/) (2022)
- [ROAD-Waymo: A Large-Scale Action Awareness Dataset for Autonomous Driving](https://mlsanity.com/paper/2411.01683/) (2024)
- [Design Insights and Comparative Evaluation of a Hardware-Based Cooperative Perception Architecture for Lane Change Prediction](https://mlsanity.com/paper/2509.20218/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [Monocular Vision-based Prediction of Cut-in Maneuvers with LSTM Networks](https://mlsanity.com/paper/2203.10707/) (2022)
- [PPD: A New Valet Parking Pedestrian Fisheye Dataset for Autonomous Driving](https://mlsanity.com/paper/2309.11002/) (2023)
- [From Real-World Traffic Data to Relevant Critical Scenarios](https://mlsanity.com/paper/2512.07482/) (2025)

## Abstract (arXiv)

In highway scenarios, an alert human driver will typically anticipate early cut-in/cut-out maneuvers of surrounding vehicles using visual cues mainly. Autonomous vehicles must anticipate these situations at an early stage too, to increase their safety and efficiency. In this work, lane-change recognition and prediction tasks are posed as video action recognition problems. Up to four different two-stream-based approaches, that have been successfully applied to address human action recognition, are adapted here by stacking visual cues from forward-looking video cameras to recognize and anticipate lane-changes of target vehicles. We study the influence of context and observation horizons on performance, and different prediction horizons are analyzed. The different models are trained and evaluated using the PREVENTION dataset. The obtained results clearly demonstrate the potential of these methodologies to serve as robust predictors of future lane-changes of surrounding vehicles proving an accuracy higher than 90% in time horizons of between 1-2 seconds.