mlsanityData to 7 Oct 2026

End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

arXiv 2108.08265 ↗First posted 18 Aug 2021cs.CV cs.ROcited by 191

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

End-to-end approaches to autonomous driving commonly rely on expert demonstrations. Although humans are good drivers, they are not good coaches for end-to-end algorithms that demand dense on-policy supervision. On the contrary, automated experts that leverage privileged information can efficiently generate large scale on-policy and off-policy demonstrations. However, existing automated experts for urban driving make heavy use of hand-crafted rules and perform suboptimally even on driving simulators, where ground-truth information is available. To address these issues, we train a reinforcement learning expert that maps bird's-eye view images to continuous low-level actions. While setting a new performance upper-bound on CARLA, our expert is also a better coach that provides informative supervision signals for imitation learning agents to learn from. Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance. Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the challenging public routes of the CARLA LeaderBoard.

Key facts. Cited by 191 papers in the map (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 41 of its 54 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

41 of 54 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  9. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  12. 2016Generative Adversarial Imitation Learningcited by 1,576
29 more
  1. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  2. 2016End to End Learning for Self-Driving Carscited by 1,000
  3. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  4. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  5. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  6. 2019Learning by Cheatingcited by 361
  7. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  8. 2018Learning to Drive in a Daycited by 266
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  10. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  11. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  12. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  13. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  14. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 147
  15. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  16. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  17. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  18. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  19. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  20. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 75
  21. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  22. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  23. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  24. 2020Label Efficient Visual Abstractions for Autonomous Drivingcited by 33
  25. 2018End-to-End Learning of Driving Models with Surround-View Cameras and Route Plannerscited by 27
  26. 2020LaND: Learning to Navigate from Disengagementscited by 24
  27. 2018Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networkscited by 19
  28. 2019Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing fieldcited by 14
  29. 2020Learning Accurate and Human-Like Driving using Semantic Maps and Attentioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 22; 2023: 43; 2024: 44; 2025: 60; 2026: 19.

191 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  4. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  5. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  6. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  7. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  8. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  9. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  10. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  11. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  12. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
38 more
  1. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  2. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  3. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  4. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  5. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  6. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
  7. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  8. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  9. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  10. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  11. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  12. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  13. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  14. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  15. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  16. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  17. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
  18. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  19. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  20. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  21. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 59
  22. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  23. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  24. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  25. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
  26. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  27. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  28. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  29. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  30. 2022Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretrainingcited by 37
  31. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  32. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  33. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  34. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  35. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  36. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  37. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  38. 2024Hidden Biases of End-to-End Driving Datasetscited by 30

50 most cited of 191 All 191 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients
  2. 2026Imitation Learning for Autonomous Driving in CARLA
  3. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  4. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
  5. 2021Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
  6. 2019Safer End-to-End Autonomous Driving via Conditional Imitation Learning and Command Augmentation
  7. 2019Urban Driving with Conditional Imitation Learning
  8. 2019Improved Reinforcement Learning through Imitation Learning Pretraining Towards Image-based Autonomous Driving