mlsanityData to 7 Oct 2026

Abductive Ego-View Accident Video Understanding for Safe Driving Perception

arXiv 2403.00436 ↗First posted 1 Mar 2024cs.CV cs.AIcited by 43

Not on the map. Why?

Abstract (arXiv)

We present MM-AU, a novel dataset for Multi-Modal Accident video Understanding. MM-AU contains 11,727 in-the-wild ego-view accident videos, each with temporally aligned text descriptions. We annotate over 2.23 million object boxes and 58,650 pairs of video-based accident reasons, covering 58 accident categories. MM-AU supports various accident understanding tasks, particularly multimodal video diffusion to understand accident cause-effect chains for safe driving. With MM-AU, we present an Abductive accident Video understanding framework for Safe Driving perception (AdVersa-SD). AdVersa-SD performs video diffusion via an Object-Centric Video Diffusion (OAVD) method which is driven by an abductive CLIP model. This model involves a contrastive interaction loss to learn the pair co-occurrence of normal, near-accident, accident frames with the corresponding text descriptions, such as accident reasons, prevention advice, and accident categories. OAVD enforces the causal region learning while fixing the content of the original frame background in video generation, to find the dominant cause-effect chain for certain accidents. Extensive experiments verify the abductive ability of AdVersa-SD and the superiority of OAVD against the state-of-the-art diffusion models. Additionally, we provide careful benchmark evaluations for object detection and accident reason answering since AdVersa-SD relies on precise object and accident reason information.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 47 of its 73 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

47 of 73 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
35 more
  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  2. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  3. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  4. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  5. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  6. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  7. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  8. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  9. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  10. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  11. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  12. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  13. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  14. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  15. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  16. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  17. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  18. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  19. 2020Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learningcited by 91
  20. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  21. 2023DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Drivingcited by 73
  22. 2019Unsupervised Traffic Accident Detection in First-Person Videoscited by 69
  23. 2019DADA: Driver Attention Prediction in Driving Accident Scenarioscited by 69
  24. 2022Video Graph Transformer for Video Question Answeringcited by 63
  25. 2019DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmarkcited by 57
  26. 2019Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolutioncited by 57
  27. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  28. 2018Anticipating Traffic Accidents with Adaptive Loss and Large-scale Incident DBcited by 51
  29. 2021DRIVE: Deep Reinforced Accident Anticipation with Visual Explanationcited by 46
  30. 2015Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecturecited by 42
  31. 2021A Dynamic Spatial-temporal Attention Network for Early Anticipation of Traffic Accidentscited by 42
  32. 2023Contrastive Video Question Answering via Video Graph Transformercited by 32
  33. 2021Spatio-Temporal Scene-Graph Embedding for Autonomous Vehicle Collision Predictioncited by 29
  34. 2018VIENA2: A Driving Anticipation Datasetcited by 19
  35. 2019Detection of Collision-Prone Vehicle Behavior at Intersections using Siamese Interaction LSTMcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 22; 2026: 18.

43 papers.

Sort:

By area:

  1. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
  2. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  3. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  4. 2025Nexar Dashcam Collision Prediction Dataset and Challengecited by 21
  5. 2025AVD2: Accident Video Diffusion for Accident Video Descriptioncited by 20
  6. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  7. 2025RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narrativescited by 15
  8. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Drivingcited by 9
  9. 2025Real-time Traffic Accident Anticipation with Feature Reusecited by 8
  10. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 8
  11. 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
  12. 2024Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Brakingcited by 5
31 more
  1. 2025Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happencited by 5
  2. 2025Ctrl-Crash: Controllable Diffusion for Realistic Car Crashescited by 5
  3. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 5
  4. 2025VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Modelscited by 5
  5. 2025BADAS: Context Aware Collision Prediction Using Real-World Dashcam Datacited by 4
  6. 2025Accident Anticipation via Temporal Occurrence Predictioncited by 4
  7. 2025When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Networkcited by 3
  8. 2025Towards Vision Zero: The TUM Traffic Accid3nD Datasetcited by 2
  9. 2025EQ-TAA: Equivariant Traffic Accident Anticipation via Diffusion-Based Accident Video Synthesiscited by 2
  10. 2025Causal-Entity Reflected Egocentric Traffic Accident Video Synthesiscited by 2
  11. 2026Understanding Real-World Traffic Safety through RoadSafe365 Benchmarkcited by 2
  12. 2026RiskProp: Collision-Anchored Self-Supervised Risk Propagation for Early Accident Anticipationcited by 2
  13. 2024Hybrid Video Anomaly Detection for Anomalous Scenarios in Autonomous Drivingcited by 1
  14. 2025What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studiescited by 1
  15. 2025MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillancecited by 1
  16. 2025CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysiscited by 1
  17. 2026UDVideoQA: A Traffic Video Question Answering Dataset for Multi-Object Spatio-Temporal Reasoning in Urban Dynamicscited by 1
  18. 2026Interpretable Traffic Responsibility from Dashcam Video via Legal Multi Agent Reasoningcited by 1
  19. 2026Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0cited by 1
  20. 2026AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Modelscited by 1
  21. 2026Driving Video Retrieval for Complex Queries with Structured Groundingcited by 1
  22. 2026A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Modelscited by 1
  23. 2026From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQAcited by 1
  24. 2025SAVeD: A First-Person Social Media Video Dataset for ADAS-equipped vehicle Near-Miss and Crash Event Analysescited by 0
  25. 2026PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Predictioncited by 0
  26. 2026Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Modelscited by 0
  27. 2026DispatchRAG: Grounding Emergency Dispatch Decisions in Real-World Protocols from Traffic Accident Videocited by 0
  28. 2026CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarioscited by 0
  29. 2026CoRE: Weakly Supervised Coarse-to-Fine Risk Evidence Learning in Driving Videoscited by 0
  30. 2026Observe Before You Alert: Adaptive Driver Alerting with Vision-Language Modelscited by 0
  31. 2026Progressive Risk Estimation for Accident Anticipationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring the Potential of Multi-Modal AI for Driving Hazard Prediction
  2. 2025Hierarchical Reasoning with Vision-Language Models for Incident Reports from Dashcam Videos
  3. 2025AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond
  4. 2026CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning
  5. 2026UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving
  6. 2025When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis
  7. 2024Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding
  8. 2025World Model-Based End-to-End Scene Generation for Accident Anticipation in Autonomous Driving