mlsanityData to 14 Oct 2026

Traffic-MLLM: Curiosity-Regularized Supervised Learning for Traffic Scenario Case-Based Reasoning

arXiv 2509.11165 ↗First posted 14 Sep 2025cs.CVcited by 2

On the map: LLM reasoning → Chain-of-thought reasoning for driving

Abstract (arXiv)

For safe and robust autonomous driving, decision-making systems must effectively leverage past experiences to handle the inherent long-tail of traffic scenarios. Case-Based Reasoning (CBR) provides a natural paradigm for this by adapting solutions from prior cases. However, in complex and dynamic traffic environments, traditional CBR methods struggle to effectively abstract and adapt knowledge under uncertainty. Meanwhile, although multimodal large language models (MLLMs) exhibit strong perceptual and linguistic capabilities, their reasoning behavior often relies on empirical pattern fitting, limiting robustness under distribution shift and long-tail scenarios. We propose Traffic-MLLM, a retrieval-free neural case modeling framework for multimodal traffic reasoning. Instead of performing explicit case retrieval at inference time, Traffic-MLLM learns a structured and generalizable case space directly during training. To support this learning process, we construct a multi-source case base by integrating dynamic traffic videos and large-scale static visual question-answering data, serving as a unified training substrate for learning structured case representations. To further improve representation quality near knowledge boundaries, we introduce a curiosity-driven refinement mechanism based on Random Network Distillation (RND), encouraging the model to internalize cross-case structural regularities rather than surface correlations. Experiments on the SUTD-TrafficQA and DriveQA benchmarks demonstrate consistent improvements in dynamic reasoning, regulatory understanding, and cross-domain transfer. Traffic-MLLM achieves 50.8% accuracy on SUTD-TrafficQA, 74.8% on the CARLA-based DriveQA split, and 83.1% on the real-world Mapillary split, indicating that representation-level case-space refinement provides an effective alternative to explicit retrieval for scalable multimodal case adaptation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 22 of its 29 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

22 of 29 references.

References · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  9. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  11. 2018Exploration by Random Network Distillationcited by 993
  12. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
10 more
  1. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  3. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 207
  4. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  5. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  6. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  7. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  8. 2024RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Drivingcited by 30
  9. 2023Tem-adapter: Adapting Image-Text Pretraining for Video Question Answercited by 22
  10. 2025DriveQA: Passing the Driving Knowledge Testcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Case-based Reasoning Augmented Large Language Model Framework for Decision Making in Realistic Safety-Critical Driving Scenarios
  2. 2022Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
  3. 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
  4. 2023A Study of Situational Reasoning for Traffic Understanding
  5. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving
  6. 2026AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models
  7. 2025Structured Prompting and Multi-Agent Knowledge Distillation for Traffic Video Interpretation and Risk Inference
  8. 2026DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving