mlsanityData to 14 Oct 2026

Traffic-Domain Video Question Answering with Automatic Captioning

arXiv 2307.09636 ↗First posted 18 Jul 2023cs.CV cs.AIcited by 10

On the map: LLM reasoning → Chain-of-thought reasoning for driving

Abstract (arXiv)

Video Question Answering (VidQA) exhibits remarkable potential in facilitating advanced machine reasoning capabilities within the domains of Intelligent Traffic Monitoring and Intelligent Transportation Systems. Nevertheless, the integration of urban traffic scene knowledge into VidQA systems has received limited attention in previous research endeavors. In this work, we present a novel approach termed Traffic-domain Video Question Answering with Automatic Captioning (TRIVIA), which serves as a weak-supervision technique for infusing traffic-domain knowledge into large video-language models. Empirical findings obtained from the SUTD-TrafficQA task highlight the substantial enhancements achieved by TRIVIA, elevating the accuracy of representative video-language models by a remarkable 6.5 points (19.88%) compared to baseline settings. This pioneering methodology holds great promise for driving advancements in the field, inspiring researchers and practitioners alike to unlock the full potential of emerging video-language models in traffic-related applications.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 37 of its 56 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

37 of 56 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  6. 2021Video Swin Transformercited by 1,091
  7. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  8. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  9. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  10. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  11. 2020A2D2: Audi Autonomous Driving Datasetcited by 190
  12. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 188
25 more
  1. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 184
  2. 2022Global Tracking Transformerscited by 110
  3. 2019ASER: A Large-scale Eventuality Knowledge Graphcited by 89
  4. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  5. 2017HoloClean: Holistic Data Repairs with Probabilistic Inferencecited by 79
  6. 2019Inducing Relational Knowledge from BERTcited by 75
  7. 2016Label-Free Supervision of Neural Networks with Physics and Domain Knowledgecited by 74
  8. 2014Curriculum Learning of Multiple Taskscited by 66
  9. 2017Neural Ranking Models with Weak Supervisioncited by 61
  10. 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
  11. 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
  12. 2021Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Trainingcited by 41
  13. 2020Temporal Common Sense Acquisition with Minimal Supervisioncited by 39
  14. 2021Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Modelcited by 39
  15. 2019Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answeringcited by 37
  16. 2020Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervisioncited by 27
  17. 2020Advanced Semantics for Commonsense Knowledge Extractioncited by 24
  18. 2022Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmarkcited by 16
  19. 2020Neuro-symbolic Architectures for Context Understandingcited by 13
  20. 2021Table-based Fact Verification with Salience-aware Learningcited by 9
  21. 2019Truck Traffic Monitoring with Satellite Imagescited by 4
  22. 2022PInKS: Preconditioned Commonsense Inference with Minimal Supervisioncited by 4
  23. 2022VIPHY: Probing "Visible" Physical Commonsense Knowledgecited by 4
  24. 2020Urban Traffic Monitoring and Modeling System: An IoT Solution for Enhancing Road Safetycited by 3
  25. 2022Intelligent Traffic Monitoring with Hybrid AIcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 5; 2026: 1.

10 papers.

  1. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  2. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  3. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  4. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  5. 2024TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioningcited by 17
  6. 2025RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narrativescited by 15
  7. 2024Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understandingcited by 10
  8. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  9. 2026Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Datasetcited by 2
  10. 2025InterAct-Video: Reasoning-Rich Video QA for Urban Trafficcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Eyes on the Road: State-of-the-Art Video Question Answering Models Assessment for Traffic Monitoring Tasks
  2. 2026UDVideoQA: A Traffic Video Question Answering Dataset for Multi-Object Spatio-Temporal Reasoning in Urban Dynamics
  3. 2025TUMTraffic-VideoQA: A Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenes
  4. 2026From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA
  5. 2026Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus
  6. 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning
  7. 2022Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
  8. 2025RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation System