mlsanityData to 7 Oct 2026

Textual Explanations for Self-Driving Vehicles

arXiv 1807.11546 ↗First posted 30 Jul 2018cs.CVcited by 226

Not on the map. Why?

Cited by 91 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep neural perception and control networks have become key components of self-driving vehicles. User acceptance is likely to benefit from easy-to-interpret textual explanations which allow end-users to understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller's output, namely rationalizations. We propose a new approach to introspective explanations which consists of two parts. First, we use a visual (spatial) attention model to train a convolutional network end-to-end from images to the vehicle control commands, i.e., acceleration and change of course. The controller's attention identifies image regions that potentially influence the network's output. Second, we use an attention-based video-to-text model to produce textual explanations of model actions. The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. Finally, we explore a version of our model that generates rationalizations, and compare with introspective explanations on the same video segments. We evaluate these models on a novel driving dataset with ground-truth human explanations, the Berkeley DeepDrive eXplanation (BDD-X) dataset. Code is available at https://github.com/JinkyuKimUCB/explainable-deep-driving.

Key facts. Cited by 226 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 26; 2023: 38; 2024: 51; 2025: 59; 2026: 29.

226 papers.

Sort:

By area:

  1. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  5. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  6. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  7. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  8. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  9. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  10. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  11. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  12. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
38 more
  1. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  2. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  4. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  5. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  6. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  7. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 125
  8. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  9. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  10. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 120
  11. 2023Language Prompt for Autonomous Drivingcited by 110
  12. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  13. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  14. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
  15. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  16. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  17. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  18. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  19. 2021Explanations in Autonomous Driving: A Surveycited by 80
  20. 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 79
  21. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  22. 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 75
  23. 2024Automated Evaluation of Large Vision-Language Models on Self-driving Corner Casescited by 75
  24. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  25. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  26. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  27. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  28. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  29. 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
  30. 2024Embodied Understanding of Driving Scenarioscited by 63
  31. 2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
  32. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  33. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 52
  34. 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
  35. 2024A Survey for Foundation Models in Autonomous Drivingcited by 51
  36. 2023HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Drivingcited by 48
  37. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  38. 2021DRIVE: Deep Reinforced Accident Anticipation with Visual Explanationcited by 46

50 most cited of 226 All 226 (JSON)