# An LLM-Explainable DRL Framework for Passenger-Directed Autonomous Driving

arXiv: https://arxiv.org/abs/2606.20640 · HTML: https://mlsanity.com/paper/2606.20640/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Placed in

- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)

## Builds on (most cited first, 10 of 10)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment](https://mlsanity.com/paper/2503.09594/) (2025)
- [RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model](https://mlsanity.com/paper/2402.10828/) (2024)
- [Personalized Autonomous Driving with Large Language Models: Field Experiments](https://mlsanity.com/paper/2312.09397/) (2023)
- [Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving](https://mlsanity.com/paper/2310.16639/) (2023)
- [X-Driver: Explainable Autonomous Driving with Vision-Language Models](https://mlsanity.com/paper/2505.05098/) (2025)
- [Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection](https://mlsanity.com/paper/2109.14337/) (2021)
- [Multimodal Framework for Explainable Autonomous Driving: Integrating Video, Sensor, and Textual Data for Enhanced Decision-Making and Transparency](https://mlsanity.com/paper/2507.07938/) (2025)

## Abstract (arXiv)

Autonomous vehicles offer the potential for safer and more efficient mobility, yet public trust remains limited due to the lack of transparency in their decision-making. This work addresses this issue by combining deep reinforcement learning (DRL) for adaptive driving control with large language model (LLM)-based explainability modules designed to communicate agent behavior to passengers. DRL agents were trained in simulation using a Dueling Double Deep Q-Network to follow distinct driving requests: textitfast, textitcomfort, and textitstop. They demonstrated stable learning, safe compliance with traffic rules, and reliable switching between modes within a single trip. In parallel, LLM modules were introduced to interpret passenger requests, determine when explanations were needed, and generate concise, safety-oriented justifications. Results show that this framework, serving as a proof of concept for integrating RL decision-making and LLMs, balances safety, adaptability, and explainability, and is most effective when requests are delayed or overridden due to safety constraints.