# Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments

arXiv: https://arxiv.org/abs/1607.02329 · HTML: https://mlsanity.com/paper/1607.02329/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2016.

## Built on by (most cited first, 30 of 33)

- [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)
- [Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses](https://mlsanity.com/paper/2104.01789/) (2021)
- [Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches](https://mlsanity.com/paper/2111.06740/) (2021)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [Event-Guided Procedure Planning from Instructional Videos with Text Supervision](https://mlsanity.com/paper/2308.08885/) (2023)
- [Imitating Language via Scalable Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.01369/) (2024)
- [Active Task-Inference-Guided Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2001.09227/) (2020)
- [Inducing Structure in Reward Learning by Learning Features](https://mlsanity.com/paper/2201.07082/) (2022)
- [Off-road Autonomous Vehicles Traversability Analysis and Trajectory Planning Based on Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1909.06953/) (2019)
- [The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior](https://mlsanity.com/paper/2107.13507/) (2021)
- [Learning Perceptual Concepts by Bootstrapping from Human Queries](https://mlsanity.com/paper/2111.05251/) (2021)
- [Massively Scalable Inverse Reinforcement Learning in Google Maps](https://mlsanity.com/paper/2305.11290/) (2023)
- [Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2201.06539/) (2022)
- [Generative Planning for Temporally Coordinated Exploration in Reinforcement Learning](https://mlsanity.com/paper/2201.09765/) (2022)
- [CREStE: Scalable Mapless Navigation with Internet Scale Priors and Counterfactual Guidance](https://mlsanity.com/paper/2503.03921/) (2025)
- [Time-Efficient Reward Learning via Visually Assisted Cluster Ranking](https://mlsanity.com/paper/2212.00169/) (2022)
- [Autonomous Driving with Deep Reinforcement Learning in CARLA Simulation](https://mlsanity.com/paper/2306.11217/) (2023)
- [Self-Supervised Simultaneous Multi-Step Prediction of Road Dynamics and Cost Map](https://mlsanity.com/paper/2103.01039/) (2021)
- [Deep Q-Network Based Decision Making for Autonomous Driving](https://mlsanity.com/paper/2303.11634/) (2023)
- [Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning](https://mlsanity.com/paper/2303.13986/) (2023)
- [Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning](https://mlsanity.com/paper/2101.00186/) (2021)
- [A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning](https://mlsanity.com/paper/2108.03222/) (2021)
- [Multi-lane Cruising Using Hierarchical Planning and Reinforcement Learning](https://mlsanity.com/paper/2110.00650/) (2021)
- [Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Search](https://mlsanity.com/paper/2202.06443/) (2022)
- [MLNav: Learning to Safely Navigate on Martian Terrains](https://mlsanity.com/paper/2203.04563/) (2022)
- [MaskPlanner: Learning-Based Object-Centric Motion Generation from 3D Point Clouds](https://mlsanity.com/paper/2502.18745/) (2025)
- [Learning from Physical Human Feedback: An Object-Centric One-Shot Adaptation Method](https://mlsanity.com/paper/2203.04951/) (2022)
- [DeePLT: Personalized Lighting Facilitates by Trajectory Prediction of Recognized Residents in the Smart Home](https://mlsanity.com/paper/2304.08027/) (2023)
- [A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach](https://mlsanity.com/paper/2512.11944/) (2025)

## Abstract (arXiv)

In this work, we present an approach to learn cost maps for driving in complex urban environments from a very large number of demonstrations of driving behaviour by human experts. The learned cost maps are constructed directly from raw sensor measurements, bypassing the effort of manually designing cost maps as well as features. When deploying the learned cost maps, the trajectories generated not only replicate human-like driving behaviour but are also demonstrably robust against systematic errors in putative robot configuration. To achieve this we deploy a Maximum Entropy based, non-linear IRL framework which uses Fully Convolutional Neural Networks (FCNs) to represent the cost model underlying expert driving behaviour. Using a deep, parametric approach enables us to scale efficiently to large datasets and complex behaviours by being run-time independent of dataset extent during deployment. We demonstrate the scalability and the performance of the proposed approach on an ambitious dataset collected over the course of one year including more than 25k demonstration trajectories extracted from over 120km of driving around pedestrianised areas in the city of Milton Keynes, UK. We evaluate the resulting cost representations by showing the advantages over a carefully manually designed cost map and, in addition, demonstrate its robustness to systematic errors by learning precise cost-maps even in the presence of system calibration perturbations.