# End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances

arXiv: https://arxiv.org/abs/1911.10868 · HTML: https://mlsanity.com/paper/1911.10868/

## Key facts

- Cited by 149 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2019.

## Built on by (most cited first, 30 of 149)

- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [LMDrive: Closed-Loop End-to-End Driving with Large Language Models](https://mlsanity.com/paper/2312.07488/) (2023)
- [VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning](https://mlsanity.com/paper/2402.13243/) (2024)
- [Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving](https://mlsanity.com/paper/2406.03877/) (2024)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving](https://mlsanity.com/paper/2310.01957/) (2023)
- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving](https://mlsanity.com/paper/2410.22313/) (2024)
- [End-to-End Driving with Online Trajectory Evaluation via BEV World Model](https://mlsanity.com/paper/2504.01941/) (2025)
- [Model-Based Imitation Learning for Urban Driving](https://mlsanity.com/paper/2210.07729/) (2022)
- [DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2308.00398/) (2023)
- [Learning from All Vehicles](https://mlsanity.com/paper/2203.11934/) (2022)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients](https://mlsanity.com/paper/2204.13683/) (2022)
- [Hidden Biases of End-to-End Driving Models](https://mlsanity.com/paper/2306.07957/) (2023)
- [COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles](https://mlsanity.com/paper/2205.02222/) (2022)
- [DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planning](https://mlsanity.com/paper/2506.06659/) (2025)
- [ReasonNet: End-to-End Driving with Temporal and Global Reasoning](https://mlsanity.com/paper/2305.10507/) (2023)

## Abstract (arXiv)

Reinforcement Learning (RL) aims at learning an optimal behavior policy from its own experiments and not rule-based control methods. However, there is no RL algorithm yet capable of handling a task as difficult as urban driving. We present a novel technique, coined implicit affordances, to effectively leverage RL for urban driving thus including lane keeping, pedestrians and vehicles avoidance, and traffic light detection. To our knowledge we are the first to present a successful RL agent handling such a complex task especially regarding the traffic light detection. Furthermore, we have demonstrated the effectiveness of our method by winning the Camera Only track of the CARLA challenge.