# Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysis

arXiv: https://arxiv.org/abs/2404.08229 · HTML: https://mlsanity.com/paper/2404.08229/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [End-to-End Dense Video Captioning with Parallel Decoding](https://mlsanity.com/paper/2108.07781/) (2021)
- [Multi-modal Dense Video Captioning](https://mlsanity.com/paper/2003.07758/) (2020)
- [Translating Videos to Natural Language Using Deep Recurrent Neural Networks](https://mlsanity.com/paper/1412.4729/) (2014)
- [A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer](https://mlsanity.com/paper/2005.08271/) (2020)
- [Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning](https://mlsanity.com/paper/1804.00100/) (2018)
- [Domain Generalization with Domain-Specific Aggregation Modules](https://mlsanity.com/paper/1809.10966/) (2018)
- [Jointly Localizing and Describing Events for Dense Video Captioning](https://mlsanity.com/paper/1804.08274/) (2018)
- [Streamlined Dense Video Captioning](https://mlsanity.com/paper/1904.03870/) (2019)
- [End-to-end Lane Detection through Differentiable Least-Squares Fitting](https://mlsanity.com/paper/1902.00293/) (2019)
- [TrafficSafetyGPT: Tuning a Pre-trained Large Language Model to a Domain-Specific Expert in Transportation Safety](https://mlsanity.com/paper/2307.15311/) (2023)
- [A Vision-based System for Traffic Anomaly Detection using Deep Learning and Decision Trees](https://mlsanity.com/paper/2104.06856/) (2021)
- [Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions](https://mlsanity.com/paper/2007.00197/) (2020)
- [Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds](https://mlsanity.com/paper/2207.12659/) (2022)
- [A Region-Based Deep Learning Approach to Automated Retail Checkout](https://mlsanity.com/paper/2204.08584/) (2022)
- [ChatGPT is on the Horizon: Could a Large Language Model be Suitable for Intelligent Traffic Safety Research and Applications?](https://mlsanity.com/paper/2303.05382/) (2023)
- [Driver Maneuver Detection and Analysis using Time Series Segmentation and Classification](https://mlsanity.com/paper/2211.06463/) (2022)

## Built on by (most cited first, 9 of 9)

- [YOLO Evolution: A Comprehensive Benchmark and Architectural Review of YOLOv12, YOLO11, and Their Previous Versions](https://mlsanity.com/paper/2411.00201/) (2024)
- [The 8th AI City Challenge](https://mlsanity.com/paper/2404.09432/) (2024)
- [ScVLM: Enhancing Vision-Language Model for Safety-Critical Event Understanding](https://mlsanity.com/paper/2410.00982/) (2024)
- [Are Vision LLMs Road-Ready? A Comprehensive Benchmark for Safety-Critical Driving Video Understanding](https://mlsanity.com/paper/2504.14526/) (2025)
- [Low-Light Image Enhancement Framework for Improved Object Detection in Fisheye Lens Datasets](https://mlsanity.com/paper/2404.10078/) (2024)
- [CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution](https://mlsanity.com/paper/2503.01632/) (2025)
- [Task-Specific Dual-Model Framework for Comprehensive Traffic Safety Video Description and Analysis](https://mlsanity.com/paper/2510.11907/) (2025)
- [PerspectiveNet: Multi-View Perception for Dynamic Scene Understanding](https://mlsanity.com/paper/2410.16824/) (2024)
- [Towards Explainable AI: Multi-Modal Transformer for Video-based Image Description Generation](https://mlsanity.com/paper/2504.16788/) (2025)

## Abstract (arXiv)

This paper introduces our solution for Track 2 in AI City Challenge 2024. The task aims to solve traffic safety description and analysis with the dataset of Woven Traffic Safety (WTS), a real-world Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding. Our solution mainly focuses on the following points: 1) To solve dense video captioning, we leverage the framework of dense video captioning with parallel decoding (PDVC) to model visual-language sequences and generate dense caption by chapters for video. 2) Our work leverages CLIP to extract visual features to more efficiently perform cross-modality training between visual and textual representations. 3) We conduct domain-specific model adaptation to mitigate domain shift problem that poses recognition challenge in video understanding. 4) Moreover, we leverage BDD-5K captioned videos to conduct knowledge transfer for better understanding WTS videos and more accurate captioning. Our solution has yielded on the test set, achieving 6th place in the competition. The open source code will be available at https://github.com/UCF-SST-Lab/AICity2024CVPRW