mlsanityData to 14 Oct 2026

Large Multimodal Agents for Intelligent Transportation Systems: Architectures, Evidence, and Deployment Challenges

arXiv 2608.08184 ↗First posted 8 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large multimodal agents (LMAs) are increasingly proposed for intelligent transportation systems (ITS), but existing studies often conflate multimodality, agency, empirical performance, and deployment readiness. This review provides an auditable evidence map of 42 primary study families released between January 2023 and 3 August 2026 within a corpus of 91 mapped sources. It distinguishes model-level, system-level, and hybrid multimodality and classifies each family by system architecture and action authority. Evidence is assessed independently through functional capability (C0-C3), validation setting (E0-E4), three evidence propositions (P1-P3), and eight methodological-concern domains (Q1-Q8). Transportation semantics (P1) are directly evaluated in 23 families and multidimensional integration (P3) in 24; 19 families directly evaluate both. Evidence reconciliation (P2) remains unresolved because no family demonstrates the complete provenance-challenge-handling-comparison-outcome chain. Fourteen families reach C3, but 13 remain at E2; only one reaches E3 and none reaches E4. Across ITS domains, LMAs are best supported for semantic interpretation, intent translation, evidence organisation, scenario authoring, explanation, and specialist-tool coordination. Numerical forecasting, optimisation, simulation fidelity, hard constraints, low-level control, safety fallback, and final authority should remain with independently verifiable specialist systems or accountable humans. The review therefore supports bounded orchestration rather than replacement and provides a matched comparative evaluation protocol and staged roadmap for accountable deployment. The living evidence repository is available at https://github.com/pangjunbiao/ITS-LMA-Review.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 80 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

45 of 80 references.

References · 45

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  4. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  5. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  6. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  7. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  8. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  9. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 277
  10. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 207
  11. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  12. 2023A Language Agent for Autonomous Drivingcited by 164
33 more
  1. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 143
  2. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  3. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  4. 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 115
  5. 2024Large Multimodal Agents: A Surveycited by 105
  6. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 93
  7. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  8. 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
  9. 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
  10. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  11. 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
  12. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Drivingcited by 51
  13. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  14. 2024ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObilitycited by 34
  15. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  16. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  17. 2024Promptable Closed-loop Traffic Simulationcited by 20
  18. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  19. 2023Large Language Models in Analyzing Crash Narratives -- A Comparative Study of ChatGPT, BARD and GPT-4cited by 16
  20. 2025V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Drivingcited by 14
  21. 2024Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigmcited by 12
  22. 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMscited by 12
  23. 2024RACP: Risk-Aware Contingency Planning with Multi-Modal Predictionscited by 10
  24. 2025GATSim: Urban Mobility Simulation with Generative Agentscited by 10
  25. 2026MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Drivingcited by 10
  26. 2024Automating Traffic Model Enhancement with AI Research Agentcited by 8
  27. 2024Using Multimodal Large Language Models for Automated Detection of Traffic Safety Critical Eventscited by 7
  28. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 6
  29. 2024TransitGPT: A Generative AI-based framework for interacting with GTFS data using Large Language Modelscited by 5
  30. 2026DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Drivingcited by 3
  31. 2021Completion and Augmentation based Spatiotemporal Deep Learning Approach for Short-Term Metro Origin-Destination Matrix Prediction under Limited Observable Datacited by 2
  32. 2025Automating the loop in traffic incident management on highwaycited by 2
  33. 2026AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Modelscited by 1

Built on by

Not cited yet.