mlsanityData to 7 Oct 2026

SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies

arXiv 2109.13602 ↗First posted 28 Sep 2021cs.RO cs.AI cs.CVcited by 68

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

In this paper we present the first safe system for full control of self-driving vehicles trained from human demonstrations and deployed in challenging, real-world, urban environments. Current industry-standard solutions use rule-based systems for planning. Although they perform reasonably well in common scenarios, the engineering complexity renders this approach incompatible with human-level performance. On the other hand, the performance of machine-learned (ML) planning solutions can be improved by simply adding more exemplar data. However, ML methods cannot offer safety guarantees and sometimes behave unpredictably. To combat this, our approach uses a simple yet effective rule-based fallback layer that performs sanity checks on an ML planner's decisions (e.g. avoiding collision, assuring physical feasibility). This allows us to leverage ML to handle complex situations while still assuring the safety, reducing ML planner-only collisions by 95%. We train our ML planner on 300 hours of expert driving demonstrations using imitation learning and deploy it along with the fallback layer in downtown San Francisco, where it takes complete control of a real vehicle and navigates a wide variety of challenging urban driving scenarios.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 24 of its 43 references that are in the map. First posted on arXiv 28 Sep 2021.

Builds on

24 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021Autonomy 2.0: Why is self-driving always 5 years away?16 Jul
  2. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observations26 May
  3. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2016End to End Learning for Self-Driving Carscited by 1,000
  5. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  6. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  7. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  8. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  9. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  10. 2019Causal Confusion in Imitation Learningcited by 286
  11. 2018Learning to Drive in a Daycited by 266
  12. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
9 more
  1. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  2. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  3. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 129
  4. 2018Baidu Apollo EM Motion Plannercited by 99
  5. 2019Urban Driving with Conditional Imitation Learningcited by 84
  6. 2018Motion Planning Networkscited by 78
  7. 2018Search-based optimal motion planning for automated drivingcited by 24
  8. 2019RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators from RL Policiescited by 17
  9. 2020PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Drivingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 14; 2024: 12; 2025: 18; 2026: 11.

68 papers.

Sort:

By area:

  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  2. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  3. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  4. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  5. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  6. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  7. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  8. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  10. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  11. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  12. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
38 more
  1. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  2. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  3. 2022Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Expertscited by 28
  4. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  5. 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
  6. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 19
  7. 2022Driving in Real Life with Inverse Reinforcement Learningcited by 18
  8. 2023CCIL: Context-conditioned imitation learning for urban drivingcited by 12
  9. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  10. 2025Enhancing Autonomous Driving Safety with Collision Scenario Integrationcited by 9
  11. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
  12. 2022Differentiable Constrained Imitation Learning for Robot Motion Planning and Controlcited by 7
  13. 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehiclescited by 7
  14. 2024Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Surveycited by 6
  15. 2024TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characterscited by 6
  16. 2023KI-PMF: Knowledge Integrated Plausible Motion Forecastingcited by 5
  17. 2025Spatial Retrieval Augmented Autonomous Drivingcited by 5
  18. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Fieldcited by 4
  19. 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learningcited by 4
  20. 2025DAP: A Discrete-token Autoregressive Planner for Autonomous Drivingcited by 4
  21. 2022Exploring the trade off between human driving imitation and safety for traffic simulationcited by 3
  22. 2023Efficient Baselines for Motion Prediction in Autonomous Drivingcited by 3
  23. 2024PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learningcited by 3
  24. 2025Hybrid Machine Learning Model with a Constrained Action Space for Trajectory Predictioncited by 3
  25. 2025HYPE: Hybrid Planning with Ego Proposal-Conditioned Predictionscited by 3
  26. 2023Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learningcited by 2
  27. 2026FeaXDrive: Feasibility-aware Trajectory-Centric Diffusion Planning for End-to-End Autonomous Drivingcited by 2
  28. 2022Exploring Map-based Features for Efficient Attention-based Vehicle Motion Predictioncited by 1
  29. 2022CW-ERM: Improving Autonomous Driving Planning with Closed-loop Weighted Empirical Risk Minimizationcited by 1
  30. 2024LitSim: A Conflict-aware Policy for Long-term Interactive Traffic Simulationcited by 1
  31. 2024Hybrid Imitation-Learning Motion Planner for Urban Drivingcited by 1
  32. 2024LHPF: Look back the History and Plan for the Future in Autonomous Drivingcited by 1
  33. 2025Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Surveycited by 1
  34. 2025CAFE-AD: Cross-Scenario Adaptive Feature Enhancement for Trajectory Planning in Autonomous Drivingcited by 1
  35. 2025TPK: Trustworthy Trajectory Prediction Integrating Prior Knowledge For Interpretability and Kinematic Feasibilitycited by 1
  36. 2025Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Makingcited by 1
  37. 2026RECTOR: Priority-Aware Rule-Based Reranking for Compliance-Aware Autonomous Driving Trajectory Selectioncited by 1
  38. 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Spacecited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety
  2. 2020Safe Trajectory Planning Using Reinforcement Learning for Self Driving
  3. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  4. 2026Towards Safer Autonomous Driving in an Open World: A Dual-Process Approach
  5. 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula
  6. 2022How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
  7. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients
  8. 2026Herding End-to-End Autonomous Driving via Neuro-Symbolic Safety Guards