# VIENA2: A Driving Anticipation Dataset

arXiv: https://arxiv.org/abs/1810.09044 · HTML: https://mlsanity.com/paper/1810.09044/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2018.

## Built on by (most cited first, 19 of 19)

- [DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving](https://mlsanity.com/paper/2304.01168/) (2023)
- [Abductive Ego-View Accident Video Understanding for Safe Driving Perception](https://mlsanity.com/paper/2403.00436/) (2024)
- [Vision-Based Traffic Accident Detection and Anticipation: A Survey](https://mlsanity.com/paper/2308.15985/) (2023)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning](https://mlsanity.com/paper/2106.13364/) (2021)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)
- [Nexar Dashcam Collision Prediction Dataset and Challenge](https://mlsanity.com/paper/2503.03848/) (2025)
- [AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving](https://mlsanity.com/paper/2405.07865/) (2024)
- [Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happen](https://mlsanity.com/paper/2505.21743/) (2025)
- [Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes](https://mlsanity.com/paper/2506.00227/) (2025)
- [Controllable Collision Scenario Generation via Collision Pattern Prediction](https://mlsanity.com/paper/2510.12206/) (2025)
- [Towards Vision Zero: The TUM Traffic Accid3nD Dataset](https://mlsanity.com/paper/2503.12095/) (2025)
- [EQ-TAA: Equivariant Traffic Accident Anticipation via Diffusion-Based Accident Video Synthesis](https://mlsanity.com/paper/2506.10002/) (2025)
- [Pedestrian Motion Prediction Using Transformer-based Behavior Clustering and Data-Driven Reachability Analysis](https://mlsanity.com/paper/2408.15250/) (2024)
- [ICPR 2024 Competition on Rider Intention Prediction](https://mlsanity.com/paper/2503.08437/) (2025)
- [AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models](https://mlsanity.com/paper/2604.20878/) (2026)
- [A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models](https://mlsanity.com/paper/2606.28757/) (2026)
- [Event-based Timestamp Image Encoding Network for Human Action Recognition and Anticipation](https://mlsanity.com/paper/2104.05145/) (2021)
- [Towards Safer and Understandable Driver Intention Prediction](https://mlsanity.com/paper/2510.09200/) (2025)

## Abstract (arXiv)

Action anticipation is critical in scenarios where one needs to react before the action is finalized. This is, for instance, the case in automated driving, where a car needs to, e.g., avoid hitting pedestrians and respect traffic lights. While solutions have been proposed to tackle subsets of the driving anticipation tasks, by making use of diverse, task-specific sensors, there is no single dataset or framework that addresses them all in a consistent manner. In this paper, we therefore introduce a new, large-scale dataset, called VIENA2, covering 5 generic driving scenarios, with a total of 25 distinct action classes. It contains more than 15K full HD, 5s long videos acquired in various driving conditions, weathers, daytimes and environments, complemented with a common and realistic set of sensor measurements. This amounts to more than 2.25M frames, each annotated with an action label, corresponding to 600 samples per action class. We discuss our data acquisition strategy and the statistics of our dataset, and benchmark state-of-the-art action anticipation techniques, including a new multi-modal LSTM architecture with an effective loss function for action anticipation in driving scenarios.