ROAD-Waymo: A Large-Scale Action Awareness Dataset for Autonomous Driving
Not on the map. Why?
Abstract (arXiv)
Autonomous Vehicle (AV) perception systems require more than simply seeing, via e.g., object detection or scene segmentation. They need a holistic understanding of what is happening within the scene for safe interaction with other road users. Few datasets exist for the purpose of developing and training algorithms to comprehend the actions of other road users. This paper presents ROAD-Waymo, an extensive dataset for the development and benchmarking of techniques for agent, action, location and event detection in road scenes, provided as a layer upon the (US) Waymo Open dataset. Considerably larger and more challenging than any existing dataset (and encompassing multiple cities), it comes with 198k annotated video frames, 54k agent tubes, 3.9M bounding boxes and a total of 12.4M labels. The integrity of the dataset has been confirmed and enhanced via a novel annotation pipeline designed for automatically identifying violations of requirements specifically designed for this dataset. As ROAD-Waymo is compatible with the original (UK) ROAD dataset, it provides the opportunity to tackle domain adaptation between real-world road scenarios in different countries within a novel benchmark: ROAD++.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 80 references that are in the map. First posted on arXiv 3 Nov 2024.
Builds on
56 of 80 references.
References · 56
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2017Mask R-CNNcited by 7,259
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2018SlowFast Networks for Video Recognitioncited by 1,862
- 2018Datasheets for Datasetscited by 1,371
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
38 more
- 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
- 2018Model Cards for Model Reportingcited by 1,068
- 2021Multiscale Vision Transformerscited by 896
- 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
- 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
- 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
- 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
- 2022YOLOv6: A Single-Stage Object Detection Framework for Industrial Applicationscited by 273
- 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 267
- 2016FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptationcited by 242
- 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
- 2020Logic Tensor Networkscited by 197
- 2021Cross-Domain Adaptive Teacher for Object Detectioncited by 125
- 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
- 2018Actor-Centric Relation Networkcited by 87
- 2023Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous drivingcited by 83
- 2021MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actionscited by 82
- 2021DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentationcited by 80
- 2017Action Tubelet Detector for Spatio-Temporal Action Localizationcited by 77
- 2022SIGMA: Semantic-complete Graph Matching for Domain Adaptive Object Detectioncited by 74
- 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
- 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
- 2019You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localizationcited by 58
- 2020TITAN: Future Forecast using Action Priorscited by 58
- 2016Online Real-time Multiple Spatiotemporal Action Localisation and Predictioncited by 57
- 2020Actions as Moving Pointscited by 55
- 2021TubeR: Tubelet Transformer for Video Action Detectioncited by 55
- 2021ROAD: The ROad event Awareness Dataset for Autonomous Drivingcited by 54
- 2021Multi-Label Classification Neural Networks with Hard Logical Constraintscited by 36
- 2016Deep Learning for Detecting Multiple Space-Time Action Tubes in Videoscited by 35
- 2016Joint Attention in Autonomous Driving (JAAD)cited by 35
- 2022Holistic Interaction Transformer Network for Action Detectioncited by 28
- 2023Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentationcited by 26
- 2023End-to-End Spatio-Temporal Action Localisation with Video Transformerscited by 20
- 2023EDAPS: Enhanced Domain-Adaptive Panoptic Segmentationcited by 17
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.