mlsanityData to 7 Oct 2026

Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey

arXiv 2307.04370 ↗First posted 10 Jul 2023cs.RO cs.AI cs.CVcited by 101

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

End-to-End driving is a promising paradigm as it circumvents the drawbacks associated with modular systems, such as their overwhelming complexity and propensity for error propagation. Autonomous driving transcends conventional traffic patterns by proactively recognizing critical events in advance, ensuring passengers' safety and providing them with comfortable transportation, particularly in highly stochastic and variable traffic settings. This paper presents a comprehensive review of the End-to-End autonomous driving stack. It provides a taxonomy of automated driving tasks wherein neural networks have been employed in an End-to-End manner, encompassing the entire driving process from perception to control, while addressing key challenges encountered in real-world applications. Recent developments in End-to-End autonomous driving are analyzed, and research is categorized based on underlying principles, methodologies, and core functionality. These categories encompass sensorial input, main and auxiliary output, learning approaches ranging from imitation to reinforcement learning, and model evaluation techniques. The survey incorporates a detailed discussion of the explainability and safety aspects. Furthermore, it assesses the state-of-the-art, identifies challenges, and explores future possibilities. We maintained the latest advancements and their corresponding open-source implementations at https://github.com/Pranav-chib/Recent-Advancements-in-End-to-End-Autonomous-Driving-using-Deep-Learning.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 120 of its 197 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

120 of 197 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun
  2. 2023Hidden Biases of End-to-End Driving Models13 Jun
  3. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
  4. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul
  5. 2022MMFN: Multi-Modal-Fusion-Net for End-to-End Driving1 Jul
  6. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline16 Jun
  7. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
  8. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observations26 May
  9. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr
  10. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 110

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  8. 2022Planning-oriented Autonomous Drivingcited by 1,068
  9. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  10. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  11. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  12. 2018Domain Adaptive Faster R-CNN for Object Detection in the Wildcited by 442
38 more
  1. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  2. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  3. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  4. 2019Learning by Cheatingcited by 361
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  6. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  7. 2018Learning to Drive in a Daycited by 266
  8. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  9. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  10. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  11. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  12. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  13. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  14. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  15. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  16. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  17. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  18. 2023Segment and Track Anythingcited by 193
  19. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  20. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  21. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  22. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  23. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  24. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  25. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  26. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  27. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  28. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  29. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
  30. 2022Learning from All Vehiclescited by 135
  31. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  32. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  33. 2021Learning to drive from a world on railscited by 114
  34. 2019Multimodal End-to-End Autonomous Drivingcited by 110
  35. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  36. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  37. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  38. 2015UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Trackingcited by 98

50 most cited of 110 All 110 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 29; 2025: 39; 2026: 24.

101 papers.

Sort:

By area:

  1. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  2. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  3. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  4. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  5. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  6. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learningcited by 40
  7. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  8. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  9. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 27
  10. 2024CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Trafficcited by 22
  11. 2025StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Drivingcited by 20
  12. 2025RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Drivingcited by 18
38 more
  1. 2024SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Drivingcited by 16
  2. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 15
  3. 2025UncAD: Towards Safe End-to-end Autonomous Driving via Online Map Uncertaintycited by 14
  4. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  5. 2024ParkingE2E: Camera-based End-to-end Parking Network, from Images to Planningcited by 12
  6. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 9
  7. 2024Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarkingcited by 8
  8. 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
  9. 2023Prospective Role of Foundation Models in Advancing Autonomous Vehiclescited by 7
  10. 2025NavigScene: Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Drivingcited by 7
  11. 2023Nav-Q: Quantum Deep Reinforcement Learning for Collision-Free Navigation of Self-Driving Carscited by 6
  12. 2024Pre-trained Transformer-Enabled Strategies with Human-Guided Fine-Tuning for End-to-end Navigation of Autonomous Vehiclescited by 6
  13. 2024Autonomous Driving Small-Scale Cars: A Survey of Recent Developmentcited by 6
  14. 2024Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunitiescited by 6
  15. 2025Out of the Shadows: Exploring a Latent Space for Neural Network Verificationcited by 6
  16. 2025FocalAD: Local Motion Planning for End-to-End Autonomous Drivingcited by 6
  17. 2024Robust Long-Range Perception Against Sensor Misalignment in Autonomous Vehiclescited by 5
  18. 2025DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulationcited by 5
  19. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  20. 2023Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learningcited by 3
  21. 2024TiCoSS: Tightening the Coupling between Semantic Segmentation and Stereo Matching within A Joint Learning Frameworkcited by 3
  22. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  23. 2024Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Featurescited by 3
  24. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Drivingcited by 3
  25. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  26. 2025Decentralized Vision-Based Autonomous Aerial Wildlife Monitoringcited by 3
  27. 2026SToRM: Supervised Token Reduction for Multi-modal LLMs toward efficient end-to-end autonomous drivingcited by 3
  28. 2026The Era of End-to-End Autonomy: Transitioning from Rule-Based Driving to Large Driving Modelscited by 3
  29. 2024Hierarchical and Decoupled BEV Perception Learning Framework for Autonomous Drivingcited by 2
  30. 2024Integrating End-to-End and Modular Driving Approaches for Online Corner Case Detection in Autonomous Drivingcited by 2
  31. 2024AI Thinking: A framework for rethinking artificial intelligence in practicecited by 2
  32. 2024From Imitation to Exploration: End-to-end Autonomous Driving based on World Modelcited by 2
  33. 2024Distribution Guidance Network for Weakly Supervised Point Cloud Semantic Segmentationcited by 2
  34. 2024Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approachcited by 2
  35. 2024Zero-shot Hazard Identification in Autonomous Driving: A Case Study on the COOOL Benchmarkcited by 2
  36. 2025RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learningcited by 2
  37. 2025AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Drivingcited by 2
  38. 2025Hexcute: A Compiler Framework for Automating Layout Synthesis in GPU Programscited by 2

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
  2. 2026Post-Training in End-to-End Autonomous Driving
  3. 2024Exploring the Causality of End-to-End Autonomous Driving
  4. 2026Learning-Based Behavior Planning for Automated Driving: Real-World Integration and Deployment
  5. 2023SuperDriverAI: Towards Design and Implementation for End-to-End Learning-based Autonomous Driving
  6. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions
  7. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  8. 2024Hierarchical End-to-End Autonomous Driving: Integrating BEV Perception with Deep Reinforcement Learning