mlsanityData to 7 Oct 2026

Autonomy 2.0: Why is self-driving always 5 years away?

arXiv 2107.08142 ↗First posted 16 Jul 2021cs.RO cs.AI cs.CVcited by 30

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

Despite the numerous successes of machine learning over the past decade (image recognition, decision-making, NLP, image synthesis), self-driving technology has not yet followed the same trend. In this paper, we study the history, composition, and development bottlenecks of the modern self-driving stack. We argue that the slow progress is caused by approaches that require too much hand-engineering, an over-reliance on road testing, and high fleet deployment costs. We observe that the classical stack has several bottlenecks that preclude the necessary scale needed to capture the long tail of rare events. To resolve these problems, we outline the principles of Autonomy 2.0, an ML-first approach to self-driving, as a viable alternative to the currently adopted state-of-the-art. This approach is based on (i) a fully differentiable AV stack trainable from human demonstrations, (ii) closed-loop data-driven reactive simulation, and (iii) large-scale, low-cost data collections as critical solutions towards scalability issues. We outline the general architecture, survey promising works in this direction and propose key challenges to be addressed by the community in the future.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 24 of its 44 references that are in the map. First posted on arXiv 16 Jul 2021.

Builds on

24 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observations26 May

Other references · 23

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  11. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  12. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
11 more
  1. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  2. 2019Causal Confusion in Imitation Learningcited by 286
  3. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  4. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  5. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  6. 2020DSGN: Deep Stereo Geometry Network for 3D Object Detectioncited by 100
  7. 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
  8. 2021M3DSSD: Monocular 3D Single Stage Object Detectorcited by 39
  9. 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
  10. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  11. 2021What data do we need for training an AV motion planner?cited by 15

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 9; 2023: 10; 2024: 6; 2025: 3.

30 papers.

Sort:

By area:

  1. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  2. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  3. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  4. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  5. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  6. 2025Robust Autonomy Emerges from Self-Playcited by 64
  7. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  8. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  9. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
  10. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
  11. 2022Critic Sequential Monte Carlocited by 16
  12. 2023Safety-Critical Scenario Generation Via Reinforcement Learning Based Editingcited by 16
18 more
  1. 2023A Diffusion-Model of Joint Interactive Navigationcited by 16
  2. 2024ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Drivingcited by 16
  3. 2024Generating Out-Of-Distribution Scenarios Using Language Modelscited by 10
  4. 2022Towards Driving-Oriented Metric for Lane Detection Modelscited by 7
  5. 2022LiDAR-as-Camera for End-to-End Drivingcited by 7
  6. 2022Testing predictive automated driving systems: lessons learned and future recommendationscited by 6
  7. 2023Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Drivingcited by 6
  8. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Fieldcited by 4
  9. 2023Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspectivecited by 4
  10. 2025RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Casecited by 4
  11. 2023Controlling Steering with Energy-Based Modelscited by 2
  12. 2024Integrating End-to-End and Modular Driving Approaches for Online Corner Case Detection in Autonomous Drivingcited by 2
  13. 2025LVLM-MPC Collaboration for Autonomous Driving: A Safety-Aware and Task-Scalable Control Architecturecited by 2
  14. 2023Human-Producible Adversarial Examplescited by 1
  15. 2022Quantity over Quality: Training an AV Motion Planner with Large Scale Commodity Vision Datacited by 0
  16. 2022Designing a Recurrent Neural Network to Learn a Motion Planner for High-Dimensional Inputscited by 0
  17. 2023Planning Automated Driving with Accident Experience Referencing and Common-sense Inferencingcited by 0
  18. 2024Path Planning based on 2D Object Bounding-boxcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023End-to-end Autonomous Driving: Challenges and Frontiers
  2. 2019Review of Learning-based Longitudinal Motion Planning for Autonomous Vehicles: Research Gaps between Self-driving and Traffic Congestion
  3. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving
  4. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform
  5. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
  6. 2024Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologies
  7. 2023Towards Sim2Real Transfer of Autonomy Algorithms using AutoDRIVE Ecosystem
  8. 2026The Era of End-to-End Autonomy: Transitioning from Rule-Based Driving to Large Driving Models