mlsanityData to 7 Oct 2026

ROAD: The ROad event Awareness Dataset for Autonomous Driving

arXiv 2102.11585 ↗First posted 23 Feb 2021cs.CV cs.AI cs.ROcited by 54

Not on the map. Why?

Abstract (arXiv)

Humans drive in a holistic fashion which entails, in particular, understanding dynamic road events and their evolution. Injecting these capabilities in autonomous vehicles can thus take situational awareness and decision making closer to human-level performance. To this purpose, we introduce the ROad event Awareness Dataset (ROAD) for Autonomous Driving, to our knowledge the first of its kind. ROAD is designed to test an autonomous vehicle's ability to detect road events, defined as triplets composed by an active agent, the action(s) it performs and the corresponding scene locations. ROAD comprises videos originally from the Oxford RobotCar Dataset annotated with bounding boxes showing the location in the image plane of each road event. We benchmark various detection tasks, proposing as a baseline a new incremental algorithm for online road event awareness termed 3D-RetinaNet. We also report the performance on the ROAD tasks of Slowfast and YOLOv5 detectors, as well as that of the winners of the ICCV2021 ROAD challenge, which highlight the challenges faced by situation awareness in autonomous driving. ROAD is designed to allow scholars to investigate exciting tasks such as complex (road) activity detection, future event anticipation and continual learning. The dataset is available at https://github.com/gurkirt/road-dataset; the baseline can be found at https://github.com/gurkirt/3D-RetinaNet.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 56 of its 121 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

56 of 121 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2017Non-local Neural Networkscited by 2,290
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
38 more
  1. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  2. 2017Focal Loss for Dense Object Detectioncited by 749
  3. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  4. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  5. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  6. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  7. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  8. 2019Human Motion Trajectory Prediction: A Surveycited by 231
  9. 2018Machine Theory of Mindcited by 222
  10. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  11. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  12. 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 171
  13. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
  14. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  15. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  16. 2018TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactionscited by 82
  17. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  18. 2017Action Tubelet Detector for Spatio-Temporal Action Localizationcited by 77
  19. 2018Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practicecited by 74
  20. 2020Asynchronous Interaction Aggregation for Action Detectioncited by 65
  21. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 64
  22. 2020TITAN: Future Forecast using Action Priorscited by 58
  23. 2016Online Real-time Multiple Spatiotemporal Action Localisation and Predictioncited by 57
  24. 2020Actions as Moving Pointscited by 55
  25. 2019STEP: Spatio-Temporal Progressive Learning for Video Action Detectioncited by 51
  26. 2014Finding Action Tubescited by 50
  27. 2015Learning to track for spatio-temporal action localizationcited by 44
  28. 2017Occam's razor is insufficient to infer the preferences of irrational agentscited by 44
  29. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  30. 2018Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systemscited by 42
  31. 2019TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detectioncited by 36
  32. 2016Deep Learning for Detecting Multiple Space-Time Action Tubes in Videoscited by 35
  33. 2019Dance with Flow: Two-in-One Stream Action Detectioncited by 30
  34. 2019D2-City: A Large-Scale Dashcam Video Dataset of Diverse Traffic Scenarioscited by 28
  35. 2018Is the Pedestrian going to Cross? Answering by 2D Pose Estimationcited by 25
  36. 2020Pedestrian Path, Pose and Intention Prediction through Gaussian Process Dynamical Models and Pedestrian Activity Recognitioncited by 20
  37. 2018A Better Baseline for AVAcited by 16
  38. 2017AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecturecited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 13; 2024: 18; 2025: 8; 2026: 4.

54 papers.

Sort:

By area:

  1. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  2. 2023OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mappingcited by 83
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  4. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  5. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 52
  6. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  7. 2022ROAD-R: The Autonomous Driving Dataset with Logical Requirementscited by 34
  8. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  9. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 22
  10. 2025RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narrativescited by 15
  11. 2022Spatio-Temporal Action Detection Under Large Motioncited by 14
  12. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
38 more
  1. 2021METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviorscited by 11
  2. 2023Explainable Action Prediction through Self-Supervision on Scene Graphscited by 9
  3. 2022From Spoken Thoughts to Automated Driving Commentary: Predicting and Explaining Intelligent Vehicles' Actionscited by 8
  4. 2023Machine Learning with Requirements: a Manifestocited by 8
  5. 2024Exploiting VLM Localizability and Semantics for Open Vocabulary Action Detectioncited by 8
  6. 2022Argus++: Robust Real-time Activity Detection for Unconstrained Video Streams with Overlapping Cube Proposalscited by 6
  7. 2021Spatiotemporal Deformable Scene Graphs for Complex Activity Detectioncited by 5
  8. 2023Towards Robust Probabilistic Modeling on SO(3) via Rotation Laplace Distributioncited by 5
  9. 2024Exploiting T-norms for Deep Learning in Autonomous Drivingcited by 4
  10. 2021Probabilistic Approach for Road-Users Detectioncited by 3
  11. 2022Improving Transfer Learning with a Dual Image and Video Transformer for Multi-label Movie Trailer Genre Classificationcited by 3
  12. 2023Temporal DINO: A Self-supervised Video Strategy to Enhance Action Predictioncited by 3
  13. 2024Self-Supervised Multi-Frame Neural Scene Flowcited by 3
  14. 2026Random-Set Graph Neural Networkscited by 3
  15. 2021Monitoring and Adapting the Physical State of a Camera for Autonomous Vehiclescited by 2
  16. 2022RALACs: Action Recognition in Autonomous Vehicles using Interaction Encoding and Optical Flowcited by 2
  17. 2023A Survey on Datasets for Decision-making of Autonomous Vehiclecited by 2
  18. 2023A Hybrid Graph Network for Complex Activity Detection in Videocited by 2
  19. 2024PiShield: A PyTorch Package for Learning with Requirementscited by 2
  20. 2024AssistTaxi: A Comprehensive Dataset for Taxiway Analysis and Autonomous Operationscited by 2
  21. 2025ATARS: An Aerial Traffic Atomic Activity Recognition and Temporal Segmentation Datasetcited by 2
  22. 2024Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learningcited by 1
  23. 2024ROAD-Waymo: A Large-Scale Action Awareness Dataset for Autonomous Drivingcited by 1
  24. 2024Harnessing Smartphone Sensors for Enhanced Road Safety: A Comprehensive Dataset and Reviewcited by 1
  25. 2025AuthSim: Towards Authentic and Effective Safety-critical Scenario Generation for Autonomous Driving Testscited by 1
  26. 2025Last Layer Hamiltonian Monte Carlocited by 1
  27. 2022Multi-Task Edge Prediction in Temporally-Dynamic Video Graphscited by 0
  28. 2023Reducing the False Positive Rate Using Bayesian Inference in Autonomous Driving Perceptioncited by 0
  29. 2023Spatiotemporal Event Graphs for Dynamic Scene Understandingcited by 0
  30. 2024High-Order Evolving Graphs for Enhanced Representation of Traffic Dynamicscited by 0
  31. 2024A low complexity contextual stacked ensemble-learning approach for pedestrian intent predictioncited by 0
  32. 2024GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulationcited by 0
  33. 2024First Place Solution to the ECCV 2024 ROAD++ Challenge @ ROAD++ Spatiotemporal Agent Detection 2024cited by 0
  34. 2024First Place Solution to the ECCV 2024 ROAD++ Challenge @ ROAD++ Atomic Activity Recognition 2024cited by 0
  35. 2024On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidancecited by 0
  36. 2025An object detection approach for lane change and overtake detection from motion profilescited by 0
  37. 2025EMT: A Visual Multi-Task Benchmark Dataset for Autonomous Drivingcited by 0
  38. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0

50 most cited of 54 All 54 (JSON)