mlsanityData to 14 Oct 2026

Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends

arXiv 2504.16134 ↗First posted 21 Apr 2025cs.CV cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

Abstract (arXiv)

Traffic safety remains a critical global challenge, with traditional Advanced Driver-Assistance Systems (ADAS) often struggling in dynamic real-world scenarios due to fragmented sensor processing and susceptibility to adversarial conditions. This paper reviews the transformative potential of Multimodal Large Language Models (MLLMs) in addressing these limitations by integrating cross-modal data such as visual, spatial, and environmental inputs to enable holistic scene understanding. Through a comprehensive analysis of MLLM-based approaches, we highlight their capabilities in enhancing perception, decision-making, and adversarial robustness, while also examining the role of key datasets (e.g., KITTI, DRAMA, ML4RoadSafety) in advancing research. Furthermore, we outline future directions, including real-time edge deployment, causality-driven reasoning, and human-AI collaboration. By positioning MLLMs as a cornerstone for next-generation traffic safety systems, this review underscores their potential to revolutionize the field, offering scalable, context-aware solutions that proactively mitigate risks and improve overall road safety.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 56 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

41 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis17 Jan
  2. 2025Vision-Language Models for Autonomous Driving: CLIP-Based Dynamic Scene Understanding9 Jan
  3. 2024ScVLM: Enhancing Vision-Language Model for Safety-Critical Event Understanding1 Oct
  4. 2024Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testing26 Sep
  5. 2024Cross-domain Few-shot In-context Learning for Enhancing Traffic Sign Recognition8 Jul

Other references · 36

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  6. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  7. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  8. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
  9. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  10. 2019Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Drivingcited by 152
  11. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 145
  12. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 126
24 more
  1. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  2. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
  3. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehiclescited by 100
  4. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  5. 2024Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsificationcited by 59
  6. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understandingcited by 59
  7. 2023TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Modelscited by 45
  8. 2021Physical Adversarial Attacks on an Aerial Imagery Object Detectorcited by 35
  9. 2024CAFuser: Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenescited by 33
  10. 2023Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projectioncited by 25
  11. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  12. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  13. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  14. 2024Visual Reasoning and Multi-Agent Approach in Multimodal Large Language Models (MLLMs): Solving TSP and mTSP Combinatorial Challengescited by 21
  15. 2022TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillancecited by 20
  16. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attackscited by 14
  17. 2023Graph Neural Networks for Road Safety Modeling: Datasets and Evaluations for Accident Analysiscited by 13
  18. 2024Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigmcited by 12
  19. 2024ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomicscited by 12
  20. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  21. 2024Probing the Robustness of Vision-Language Pretrained Models: A Multimodal Adversarial Attack Approachcited by 7
  22. 2024Leveraging Large Language Models (LLMs) for Traffic Management at Urban Intersections: The Case of Mixed Traffic Scenarioscited by 6
  23. 2025On Fairness of Unified Multimodal Large Language Model for Image Generationcited by 6
  24. 2023TrafficMOT: A Challenging Dataset for Multi-Object Tracking in Complex Traffic Scenarioscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraintcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Using Multimodal Large Language Models for Automated Detection of Traffic Safety Critical Events
  2. 2026Enhancing Multimodal Large Language Models for Safety-Critical Driving Video Analysis
  3. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Review
  4. 2025DriveSOTIF: Advancing Perception SOTIF Through Multimodal Large Language Models
  5. 2024MLLM-SUL: Multimodal Large Language Model for Semantic Scene Understanding and Localization in Traffic Scenarios
  6. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understanding
  7. 2025Investigating Traffic Accident Detection Using Multimodal Large Language Models
  8. 2025Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach