mlsanityData to 7 Oct 2026

Urban Driving with Conditional Imitation Learning

arXiv 1912.00177 ↗First posted 30 Nov 2019cs.CV cs.AI cs.LGcited by 84

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 40 references that are in the map. First posted on arXiv 30 Nov 2019.

Builds on

24 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst7 Dec

Other references · 23

  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  5. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  8. 2018Self-Attention Generative Adversarial Networkscited by 809
  9. 2018Learning Dexterous In-Hand Manipulationcited by 643
  10. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  11. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  12. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
11 more
  1. 2019Causal Confusion in Imitation Learningcited by 286
  2. 2018Learning to Drive in a Daycited by 266
  3. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  4. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  5. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 75
  6. 2017Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Carcited by 65
  7. 2018On Offline Evaluation of Vision-based Driving Modelscited by 62
  8. 2018Learning to Drive from Simulation without Real World Labelscited by 39
  9. 2017Domain Randomization and Generative Models for Robotic Graspingcited by 34
  10. 2018Variational End-to-End Navigation and Localizationcited by 34
  11. 2018End-to-End Learning of Driving Models with Surround-View Cameras and Route Plannerscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 17; 2022: 18; 2023: 12; 2024: 16; 2025: 12; 2026: 8.

84 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  3. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  4. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  5. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  6. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  7. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  8. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  10. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  11. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
  12. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
38 more
  1. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  2. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  3. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Drivingcited by 46
  4. 2022SIND: A Drone Dataset at Signalized Intersection in Chinacited by 29
  5. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  6. 2022Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Expertscited by 28
  7. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  8. 2021Efficient and Robust LiDAR-Based End-to-End Navigationcited by 22
  9. 2021Learning by Watchingcited by 22
  10. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  11. 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17
  12. 2022SelfD: Self-Learning Large-Scale Driving Policies From the Webcited by 16
  13. 2022Critic Sequential Monte Carlocited by 16
  14. 2022Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixelscited by 12
  15. 2024Autonomous Driving with Spiking Neural Networkscited by 12
  16. 2020PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Drivingcited by 11
  17. 2021Reimagining an autonomous vehiclecited by 11
  18. 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Drivingcited by 11
  19. 2025Dream to Drive with Predictive Individual World Modelcited by 11
  20. 2022Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Drivingcited by 10
  21. 2025AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Drivingcited by 10
  22. 2023Learning to Drive Anywherecited by 9
  23. 2025DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Drivingcited by 9
  24. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  25. 2022LiDAR-as-Camera for End-to-End Drivingcited by 7
  26. 2023Goal-Guided Transformer-Enabled Reinforcement Learning for Efficient Autonomous Navigationcited by 7
  27. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Reviewcited by 7
  28. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
  29. 2024Predictive Coding for Decision Transformercited by 6
  30. 2025VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-grounded Autonomous Drivingcited by 6
  31. 2021Increasing the Efficiency of Policy Learning for Autonomous Vehicles by Multi-Task Representation Learningcited by 5
  32. 2021A Versatile and Efficient Reinforcement Learning Framework for Autonomous Drivingcited by 5
  33. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environmentscited by 5
  34. 2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Makingcited by 5
  35. 2025Spatial Retrieval Augmented Autonomous Drivingcited by 5
  36. 2026HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 5
  37. 2022Human-AI Shared Control via Policy Dissectioncited by 4
  38. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Fieldcited by 4

50 most cited of 84 All 84 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Dynamic Conditional Imitation Learning for Autonomous Driving
  2. 2018Conditional Affordance Learning for Driving in Urban Environments
  3. 2019Conditional Vehicle Trajectories Prediction in CARLA Urban Environment
  4. 2025Learning to Drive by Imitating Surrounding Vehicles
  5. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  6. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  7. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety
  8. 2022Multi-Task Conditional Imitation Learning for Autonomous Navigation at Crowded Intersections