# Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions

arXiv: https://arxiv.org/abs/2112.11561 · HTML: https://mlsanity.com/paper/2112.11561/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 96 of its 361 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 30 of 96)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)

## Built on by (most cited first, 30 of 73)

- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving](https://mlsanity.com/paper/2310.03026/) (2023)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://mlsanity.com/paper/2407.17140/) (2024)
- ["Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction](https://mlsanity.com/paper/2210.03735/) (2022)
- [Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review](https://mlsanity.com/paper/2402.10086/) (2024)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning](https://mlsanity.com/paper/2404.06345/) (2024)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors](https://mlsanity.com/paper/2305.11239/) (2023)
- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [Explaining Autonomous Driving Actions with Visual Question Answering](https://mlsanity.com/paper/2307.10408/) (2023)
- [A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors](https://mlsanity.com/paper/2302.13137/) (2023)
- [DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Models](https://mlsanity.com/paper/2503.11265/) (2025)
- [CREW: Facilitating Human-AI Teaming Research](https://mlsanity.com/paper/2408.00170/) (2024)
- [Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?](https://mlsanity.com/paper/2302.10766/) (2023)
- [HighwayLLM: Decision-Making and Navigation in Highway Driving with RL-Informed Language Model](https://mlsanity.com/paper/2405.13547/) (2024)
- [Large-scale moral machine experiment on large language models](https://mlsanity.com/paper/2411.06790/) (2024)
- [A Survey on Human-AI Collaboration with Large Foundation Models](https://mlsanity.com/paper/2403.04931/) (2024)
- [Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving](https://mlsanity.com/paper/2403.12176/) (2024)
- [Advancing Explainable Autonomous Vehicle Systems: A Comprehensive Review and Research Roadmap](https://mlsanity.com/paper/2404.00019/) (2024)
- [Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability](https://mlsanity.com/paper/2504.12320/) (2025)
- [DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving](https://mlsanity.com/paper/2409.10330/) (2024)
- [H-MBA: Hierarchical MamBa Adaptation for Multi-Modal Video Understanding in Autonomous Driving](https://mlsanity.com/paper/2501.04302/) (2025)
- [Empirical Optimal Risk to Quantify Model Trustworthiness for Failure Detection](https://mlsanity.com/paper/2308.03179/) (2023)
- [ADRD: LLM-Driven Autonomous Driving Based on Rule-based Decision Systems](https://mlsanity.com/paper/2506.14299/) (2025)
- [Navigating Uncertainty: The Role of Short-Term Trajectory Prediction in Autonomous Vehicle Safety](https://mlsanity.com/paper/2307.05288/) (2023)
- [ShapG: new feature importance method based on the Shapley value](https://mlsanity.com/paper/2407.00506/) (2024)
- [SegXAL: Explainable Active Learning for Semantic Segmentation in Driving Scene Scenarios](https://mlsanity.com/paper/2408.04482/) (2024)
- [TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models](https://mlsanity.com/paper/2509.04809/) (2025)
- [CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects](https://mlsanity.com/paper/2510.14904/) (2025)

## Abstract (arXiv)

Autonomous driving has achieved significant milestones in research and development over the last two decades. There is increasing interest in the field as the deployment of autonomous vehicles (AVs) promises safer and more ecologically friendly transportation systems. With the rapid progress in computationally powerful artificial intelligence (AI) techniques, AVs can sense their environment with high precision, make safe real-time decisions, and operate reliably without human intervention. However, intelligent decision-making in such vehicles is not generally understandable by humans in the current state of the art, and such deficiency hinders this technology from being socially acceptable. Hence, aside from making safe real-time decisions, AVs must also explain their AI-guided decision-making process in order to be regulatory compliant across many jurisdictions. Our study sheds comprehensive light on the development of explainable artificial intelligence (XAI) approaches for AVs. In particular, we make the following contributions. First, we provide a thorough overview of the state-of-the-art and emerging approaches for XAI-based autonomous driving. We then propose a conceptual framework that considers the essential elements for explainable end-to-end autonomous driving. Finally, we present XAI-based prospective directions and emerging paradigms for future directions that hold promise for enhancing transparency, trustworthiness, and societal acceptance of AVs.