# Learning Accurate and Human-Like Driving using Semantic Maps and Attention

arXiv: https://arxiv.org/abs/2007.07218 · HTML: https://mlsanity.com/paper/2007.07218/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization](https://mlsanity.com/paper/2103.01882/) (2021)
- [Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games](https://mlsanity.com/paper/2303.02160/) (2023)
- [Diff-Net: Image Feature Difference based High-Definition Map Change Detection for Autonomous Driving](https://mlsanity.com/paper/2107.07030/) (2021)
- [A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving](https://mlsanity.com/paper/2110.11573/) (2021)
- [Goal-constrained Sparse Reinforcement Learning for End-to-End Driving](https://mlsanity.com/paper/2103.09189/) (2021)
- [Controlling Steering with Energy-Based Models](https://mlsanity.com/paper/2301.12264/) (2023)
- [An Overview about Emerging Technologies of Autonomous Driving](https://mlsanity.com/paper/2306.13302/) (2023)
- [Yaw-Guided Imitation Learning for Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2111.06017/) (2021)

## Abstract (arXiv)

This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. To tackle the first issue we exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. The maps are used in an attention mechanism that promotes segmentation confidence masks, thus focusing the network on semantic classes in the image that are important for the current driving situation. Human-like driving is achieved using adversarial learning, by not only minimizing the imitation loss with respect to the human driver but by further defining a discriminator, that forces the driving model to produce action sequences that are human-like. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data. Extensive experiments show that our driving models are more accurate and behave more human-like than previous methods.