# DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks

arXiv: https://arxiv.org/abs/2108.05030 · HTML: https://mlsanity.com/paper/2108.05030/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 39 of its 61 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 30 of 39)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [Noisy Networks for Exploration](https://mlsanity.com/paper/1706.10295/) (2017)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Mastering Complex Control in MOBA Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1912.09729/) (2019)
- [Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations](https://mlsanity.com/paper/2008.05930/) (2020)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving](https://mlsanity.com/paper/2010.09776/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)
- [On Offline Evaluation of Vision-based Driving Models](https://mlsanity.com/paper/1809.04843/) (2018)
- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety](https://mlsanity.com/paper/1903.00640/) (2019)
- [Social Attention for Autonomous Decision-Making in Dense Traffic](https://mlsanity.com/paper/1911.12250/) (2019)
- [Driving in Dense Traffic with Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.06710/) (2019)
- [Robot Navigation in Crowds by Graph Convolutional Networks with Attention Learned from Human Gaze](https://mlsanity.com/paper/1909.10400/) (2019)
- [openDD: A Large-Scale Roundabout Drone Dataset](https://mlsanity.com/paper/2007.08463/) (2020)

## Built on by (most cited first, 13 of 13)

- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving](https://mlsanity.com/paper/2208.12263/) (2022)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning](https://mlsanity.com/paper/2408.17005/) (2024)
- [Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation](https://mlsanity.com/paper/2203.09952/) (2022)
- [Imagination-Augmented Hierarchical Reinforcement Learning for Safe and Interactive Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2311.10309/) (2023)
- [Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2406.13223/) (2024)
- [Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment](https://mlsanity.com/paper/2407.08932/) (2024)
- [A Conflicts-free, Speed-lossless KAN-based Reinforcement Learning Decision System for Interactive Driving in Roundabouts](https://mlsanity.com/paper/2408.08242/) (2024)
- [Vision-Integrated LLMs for Autonomous Driving Assistance : Human Performance Comparison and Trust Evaluation](https://mlsanity.com/paper/2502.06843/) (2025)
- [Cooperative Autonomous Driving in Diverse Behavioral Traffic: A Heterogeneous Graph Reinforcement Learning Approach](https://mlsanity.com/paper/2509.25751/) (2025)
- [Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections](https://mlsanity.com/paper/2607.21488/) (2026)
- [DiDrive: A Risk-Aware Hierarchical Diffusion Framework for Safe Offline Reinforcement Learning in Autonomous Driving](https://mlsanity.com/paper/2609.01609/) (2026)

## Abstract (arXiv)

Autonomous driving in multi-agent dynamic traffic scenarios is challenging: the behaviors of road users are uncertain and are hard to model explicitly, and the ego-vehicle should apply complicated negotiation skills with them, such as yielding, merging and taking turns, to achieve both safe and efficient driving in various settings. Traditional planning methods are largely rule-based and scale poorly in these complex dynamic scenarios, often leading to reactive or even overly conservative behaviors. Therefore, they require tedious human efforts to maintain workability. Recently, deep learning-based methods have shown promising results with better generalization capability but less hand engineering efforts. However, they are either implemented with supervised imitation learning (IL), which suffers from dataset bias and distribution mismatch issues, or are trained with deep reinforcement learning (DRL) but focus on one specific traffic scenario. In this work, we propose DQ-GAT to achieve scalable and proactive autonomous driving, where graph attention-based networks are used to implicitly model interactions, and deep Q-learning is employed to train the network end-to-end in an unsupervised manner. Extensive experiments in a high-fidelity driving simulator show that our method achieves higher success rates than previous learning-based methods and a traditional rule-based method, and better trades off safety and efficiency in both seen and unseen scenarios. Moreover, qualitative results on a trajectory dataset indicate that our learned policy can be transferred to the real world for practical applications with real-time speeds. Demonstration videos are available at https://caipeide.github.io/dq-gat/.