Task-Specific Dual-Model Framework for Comprehensive Traffic Safety Video Description and Analysis
Not on the map. Why?
Abstract (arXiv)
Traffic safety analysis requires complex video understanding to capture fine-grained behavioral patterns and generate comprehensive descriptions for accident prevention. In this work, we present a unique dual-model framework that strategically utilizes the complementary strengths of VideoLLaMA and Qwen2.5-VL through task-specific optimization to address this issue. The core insight behind our approach is that separating training for captioning and visual question answering (VQA) tasks minimizes task interference and allows each model to specialize more effectively. Experimental results demonstrate that VideoLLaMA is particularly effective in temporal reasoning, achieving a CIDEr score of 1.1001, while Qwen2.5-VL excels in visual understanding with a VQA accuracy of 60.80%. Through extensive experiments on the WTS dataset, our method achieves an S2 score of 45.7572 in the 2025 AI City Challenge Track 2, placing 10th on the challenge leaderboard. Ablation studies validate that our separate training strategy outperforms joint training by 8.6% in VQA accuracy while maintaining captioning quality.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 13 Oct 2025.
Builds on
26 of 45 references.
References · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
14 more
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
- 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 70
- 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steeringcited by 56
- 2024VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenescited by 22
- 2024TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioningcited by 17
- 2024WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understandingcited by 17
- 2024ScVLM: Enhancing Vision-Language Model for Safety-Critical Event Understandingcited by 13
- 2020From Data to Actions in Intelligent Transportation Systems: a Prescription of Functional Requirements for Model Actionabilitycited by 9
- 2024Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysiscited by 9
- 2025Leveraging Multimodal-LLMs Assisted by Instance Segmentation for Intelligent Traffic Monitoringcited by 5
- 2024Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Surveycited by 4
- 2023Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Promptscited by 3
- 2023Safety in Traffic Management Systems: A Comprehensive Surveycited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025A Unified Detection Pipeline for Robust Object Detection in Fisheye-Based Traffic Surveillancecited by 0
- 2025A Contextual Analysis of Driver-Facing and Dual-View Video Inputs for Distraction Detection in Naturalistic Driving Environmentscited by 0
- 2026Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decompositioncited by 0
- 2026Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decomposition6 May 2026
- 2025A Contextual Analysis of Driver-Facing and Dual-View Video Inputs for Distraction Detection in Naturalistic Driving Environments23 Dec 2025
- 2025A Unified Detection Pipeline for Robust Object Detection in Fisheye-Based Traffic Surveillance22 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA
- 2025CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis
- 2024CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario
- 2025When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis
- 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understanding
- 2025AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond
- 2026Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus
- 2025Hierarchical Reasoning with Vision-Language Models for Incident Reports from Dashcam Videos