mlsanityData to 7 Oct 2026

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst

arXiv 1812.03079 ↗First posted 7 Dec 2018cs.RO cs.CV cs.LGcited by 391

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert's driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress -- the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the ChauffeurNet model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world.

Key facts. Cited by 391 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 7 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 5; 2020: 6; 2021: 73; 2022: 64; 2023: 80; 2024: 60; 2025: 61; 2026: 42.

391 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  5. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  6. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  7. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  8. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  9. 2019Causal Confusion in Imitation Learningcited by 286
  10. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  11. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  12. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
38 more
  1. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  2. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  3. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  4. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  5. 2021Multimodal Virtual Point 3D Detectioncited by 166
  6. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  7. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  8. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  9. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  10. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  11. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  12. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  13. 2022Learning from All Vehiclescited by 135
  14. 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
  15. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  16. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  17. 2021Learning to drive from a world on railscited by 114
  18. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  19. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  20. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  21. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  22. 2021Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Imagescited by 98
  23. 2022COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehiclescited by 95
  24. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  25. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  26. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  27. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
  28. 2023The Waymo Open Sim Agents Challengecited by 86
  29. 2019Urban Driving with Conditional Imitation Learningcited by 84
  30. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  31. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  32. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  33. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  34. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  35. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  36. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  37. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  38. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68

50 most cited of 391 All 391 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  2. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
  3. 2019Exploring the Limitations of Behavior Cloning for Autonomous Driving
  4. 2024IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Driving
  5. 2026Imitation Learning for Autonomous Driving in CARLA
  6. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving
  7. 2024Good Data Is All Imitation Learning Needs
  8. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments