mlsanityData to 7 Oct 2026

SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events

arXiv 2103.15538 ↗First posted 29 Mar 2021cs.CVcited by 84

Not on the map. Why?

Cited by 22 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Traffic event cognition and reasoning in videos is an important task that has a wide range of applications in intelligent transportation, assisted driving, and autonomous vehicles. In this paper, we create a novel dataset, SUTD-TrafficQA (Traffic Question Answering), which takes the form of video QA based on the collected 10,080 in-the-wild videos and annotated 62,535 QA pairs, for benchmarking the cognitive capability of causal inference and event understanding models in complex traffic scenarios. Specifically, we propose 6 challenging reasoning tasks corresponding to various traffic scenarios, so as to evaluate the reasoning capability over different kinds of complex yet practical traffic events. Moreover, we propose Eclipse, a novel Efficient glimpse network via dynamic inference, in order to achieve computation-efficient and reliable video reasoning. The experiments show that our method achieves superior performance while reducing the computation cost significantly. The project page: https://github.com/SUTDCV/SUTD-TrafficQA.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

47 of 69 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  10. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  11. 2019Green AIcited by 620
  12. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
35 more
  1. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  2. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  3. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  4. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  5. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  6. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  7. 2016Image Captioning with Semantic Attentioncited by 260
  8. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  9. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  10. 2016Movie Descriptioncited by 180
  11. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  12. 2020In Defense of Grid Features for Visual Question Answeringcited by 155
  13. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  14. 2017Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representationscited by 149
  15. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  16. 2015Exploring Models and Data for Image Question Answeringcited by 139
  17. 2018AdaFrame: Adaptive Frame Selection for Fast Video Recognitioncited by 134
  18. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 133
  19. 2016Spatially Adaptive Computation Time for Residual Networkscited by 118
  20. 2017DeepStory: Video Story QA by Deep Embedded Memory Networkscited by 67
  21. 2019LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognitioncited by 65
  22. 2016End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answeringcited by 60
  23. 2016Leveraging Video Descriptions to Learn Video Question Answeringcited by 55
  24. 2019KnowIT VQA: Answering Knowledge-Based Questions about Videoscited by 53
  25. 2016MarioQA: Answering Questions by Watching Gameplay Videoscited by 42
  26. 2016A dataset and exploration of models for understanding video data through fill-in-the-blank question-answeringcited by 41
  27. 2019Video Relationship Reasoning using Gated Spatio-Temporal Energy Graphcited by 41
  28. 2020Modality Shifting Attention Network for Multi-modal Video Question Answeringcited by 38
  29. 2019Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMscited by 34
  30. 2019Efficient Video Classification Using Fewer Framescited by 25
  31. 2018Multimodal Dual Attention Memory for Video Story Question Answeringcited by 20
  32. 2018Focal Visual-Text Attention for Visual Question Answeringcited by 19
  33. 2015Sparsifying Neural Network Connections for Face Recognitioncited by 15
  34. 2020Knowledge-Based Video Question Answering with Unsupervised Scene Descriptionscited by 15
  35. 2018Movie Question Answering: Remembering the Textual Cues for Layered Visual Contentscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 19; 2024: 20; 2025: 19; 2026: 16.

84 papers.

Sort:

By area:

  1. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  2. 2024Long Context Transfer from Language to Visioncited by 502
  3. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  4. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  5. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  6. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  7. 2024VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understandingcited by 113
  8. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  9. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  10. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 73
  11. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  12. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
38 more
  1. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  2. 2024InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactionscited by 48
  3. 2024Abductive Ego-View Accident Video Understanding for Safe Driving Perceptioncited by 43
  4. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  5. 2023Vision-Based Traffic Accident Detection and Anticipation: A Surveycited by 31
  6. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  7. 2022Causal Reasoning Meets Visual Representation Learning: A Prospective Studycited by 24
  8. 2024Diff-Tracker: Text-to-Image Diffusion Models are Unsupervised Trackerscited by 24
  9. 2023Reading Between the Lanes: Text VideoQA on the Roadcited by 23
  10. 2023Tem-adapter: Adapting Image-Text Pretraining for Video Question Answercited by 22
  11. 2021Uncovering Main Causalities for Long-tailed Information Extractioncited by 21
  12. 2024CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Processcited by 20
  13. 2025AVD2: Accident Video Diffusion for Accident Video Descriptioncited by 20
  14. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  15. 2024DAM: Dynamic Adapter Merging for Continual Video QA Learningcited by 18
  16. 2022Video Question Answering with Iterative Video-Text Co-Tokenizationcited by 16
  17. 2022Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmarkcited by 16
  18. 2023Explaining Autonomous Driving Actions with Visual Question Answeringcited by 16
  19. 2024GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Eventscited by 16
  20. 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoningcited by 16
  21. 2025RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narrativescited by 15
  22. 2023Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMscited by 14
  23. 2023A Study of Situational Reasoning for Traffic Understandingcited by 13
  24. 2023Traffic-Domain Video Question Answering with Automatic Captioningcited by 10
  25. 2024Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understandingcited by 10
  26. 2025STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Modelscited by 9
  27. 2023ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videoscited by 8
  28. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  29. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 8
  30. 2025An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubescited by 7
  31. 2025V3LMA: Visual 3D-enhanced Language Model for Autonomous Drivingcited by 7
  32. 2024Eyes on the Road: State-of-the-Art Video Question Answering Models Assessment for Traffic Monitoring Taskscited by 6
  33. 2023Benchmarking Sequential Visual Input Reasoning and Prediction in Multimodal Large Language Modelscited by 5
  34. 2024Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answeringcited by 5
  35. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 5
  36. 2023Exploring the Potential of Multi-Modal AI for Driving Hazard Predictioncited by 4
  37. 2024Answering Diverse Questions via Text Attached with Key Audio-Visual Cluescited by 4
  38. 2024Hierarchical Banzhaf Interaction for General Video-Language Representation Learningcited by 4

50 most cited of 84 All 84 (JSON)