# Conditional Affordance Learning for Driving in Urban Environments

arXiv: https://arxiv.org/abs/1806.06498 · HTML: https://mlsanity.com/paper/1806.06498/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 18 Jun 2018.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Built on by (most cited first, 30 of 70)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking](https://mlsanity.com/paper/2406.15349/) (2024)
- [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023)
- [A Language Agent for Autonomous Driving](https://mlsanity.com/paper/2311.10813/) (2023)
- [Learning from All Vehicles](https://mlsanity.com/paper/2203.11934/) (2022)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023)
- [KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients](https://mlsanity.com/paper/2204.13683/) (2022)
- [Hidden Biases of End-to-End Driving Models](https://mlsanity.com/paper/2306.07957/) (2023)
- [COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles](https://mlsanity.com/paper/2205.02222/) (2022)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [ADAPT: Action-aware Driving Caption Transformer](https://mlsanity.com/paper/2302.00673/) (2023)
- [Embodied Understanding of Driving Scenarios](https://mlsanity.com/paper/2403.04593/) (2024)
- [Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety](https://mlsanity.com/paper/1903.00640/) (2019)
- [Multi-task Learning with Attention for End-to-end Autonomous Driving](https://mlsanity.com/paper/2104.10753/) (2021)
- [iPad: Iterative Proposal-centric End-to-End Autonomous Driving](https://mlsanity.com/paper/2505.15111/) (2025)
- [Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?](https://mlsanity.com/paper/2404.07569/) (2024)
- [Rethinking Closed-loop Training for Autonomous Driving](https://mlsanity.com/paper/2306.15713/) (2023)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [OCTET: Object-aware Counterfactual Explanations](https://mlsanity.com/paper/2211.12380/) (2022)
- [Causal Navigation by Continuous-time Neural Networks](https://mlsanity.com/paper/2106.08314/) (2021)
- [Learning by Watching](https://mlsanity.com/paper/2106.05966/) (2021)
- [Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances](https://mlsanity.com/paper/2502.16804/) (2025)
- [Deep Structured Reactive Planning](https://mlsanity.com/paper/2101.06832/) (2021)
- [Coaching a Teachable Student](https://mlsanity.com/paper/2306.10014/) (2023)

## Abstract (arXiv)

Most existing approaches to autonomous driving fall into one of two categories: modular pipelines, that build an extensive model of the environment, and imitation learning approaches, that map images directly to control outputs. A recently proposed third paradigm, direct perception, aims to combine the advantages of both by using a neural network to learn appropriate low-dimensional intermediate representations. However, existing direct perception approaches are restricted to simple highway situations, lacking the ability to navigate intersections, stop at traffic lights or respect speed limits. In this work, we propose a direct perception approach which maps video input to intermediate representations suitable for autonomous navigation in complex urban environments given high-level directional inputs. Compared to state-of-the-art reinforcement and conditional imitation learning approaches, we achieve an improvement of up to 68 % in goal-directed navigation on the challenging CARLA simulation benchmark. In addition, our approach is the first to handle traffic lights and speed signs by using image-level labels only, as well as smooth car-following, resulting in a significant reduction of traffic accidents in simulation.