mlsanityData to 7 Oct 2026

RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narratives

arXiv 2503.21459 ↗First posted 27 Mar 2025cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

We introduce RoadSocial, a large-scale, diverse VideoQA dataset tailored for generic road event understanding from social media narratives. Unlike existing datasets limited by regional bias, viewpoint bias and expert-driven annotations, RoadSocial captures the global complexity of road events with varied geographies, camera viewpoints (CCTV, handheld, drones) and rich social discourse. Our scalable semi-automatic annotation framework leverages Text LLMs and Video LLMs to generate comprehensive question-answer pairs across 12 challenging QA tasks, pushing the boundaries of road event understanding. RoadSocial is derived from social media videos spanning 14M frames and 414K social comments, resulting in a dataset with 13.2K videos, 674 tags and 260K high-quality QA pairs. We evaluate 18 Video LLMs (open-source and proprietary, driving-specific and general-purpose) on our road event understanding benchmark. We also demonstrate RoadSocial's utility in improving road event understanding capabilities of general-purpose Video LLMs.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 33 of its 37 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

33 of 37 references.

References · 33

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  7. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  8. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  9. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  10. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  11. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  12. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
21 more
  1. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  2. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  3. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  4. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
  5. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  6. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  7. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 136
  8. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 125
  9. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  10. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  11. 2020Explainable Object-induced Action Decision for Autonomous Vehiclescited by 88
  12. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  13. 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
  14. 2024Embodied Understanding of Driving Scenarioscited by 63
  15. 2021ROAD: The ROad event Awareness Dataset for Autonomous Drivingcited by 54
  16. 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
  17. 2024Abductive Ego-View Accident Video Understanding for Safe Driving Perceptioncited by 43
  18. 2024WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understandingcited by 17
  19. 2024IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Trafficcited by 13
  20. 2021METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviorscited by 11
  21. 2023Traffic-Domain Video Question Answering with Automatic Captioningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2025STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Modelscited by 9
  2. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Drivingcited by 9
  3. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 5
  4. 2025VL4Gaze: Unleashing Vision-Language Models for Gaze Followingcited by 3
  5. 2025Seeing before Observable: Potential Risk Reasoning in Autonomous Driving via Vision Language Modelscited by 2
  6. 2026Understanding Real-World Traffic Safety through RoadSafe365 Benchmarkcited by 2
  7. 2025MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillancecited by 1
  8. 2026CL-VISTA: Benchmarking Continual Learning in Video Large Language Modelscited by 1
  9. 2026Distorted or Fabricated? A Survey on Hallucination in Video LLMscited by 1
  10. 2026AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Modelscited by 1
  11. 2026DriveSafe: A Framework for Risk Detection and Safety Suggestions in Driving Scenarioscited by 0
  12. 2026RoadTones: Tone Controllable Text Generation from Road Event Videoscited by 0
3 more
  1. 2026PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Predictioncited by 0
  2. 2026KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptationcited by 0
  3. 2026ReVA: A Scene-Centric Dataset Beyond Repetition for Remote Sensing Video Question Answeringcited by 0