mlsanityData to 7 Oct 2026

On Offline Evaluation of Vision-based Driving Models

arXiv 1809.04843 ↗First posted 13 Sep 2018cs.CVcited by 62

Not on the map. Why?

Cited by 30 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autonomous driving models should ideally be evaluated by deploying them on a fleet of physical vehicles in the real world. Unfortunately, this approach is not practical for the vast majority of researchers. An attractive alternative is to evaluate models offline, on a pre-collected validation dataset with ground truth annotation. In this paper, we investigate the relation between various online and offline metrics for evaluation of autonomous driving models. We find that offline prediction error is not necessarily correlated with driving quality, and two models with identical prediction error can differ dramatically in their driving performance. We show that the correlation of offline evaluation with driving quality can be significantly improved by selecting an appropriate validation dataset and suitable offline metrics. The supplementary video can be viewed at https://www.youtube.com/watch?v=P8K8Z-iF0cY

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 14; 2022: 4; 2023: 18; 2024: 7; 2025: 9; 2026: 8.

62 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  3. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  4. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  5. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  6. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  7. 2019Urban Driving with Conditional Imitation Learningcited by 84
  8. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  9. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 61
  10. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  11. 2021Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systemscited by 36
  12. 2023Diversifying the High-level Features for better Adversarial Transferabilitycited by 34
38 more
  1. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 34
  2. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  3. 2021Can Offline Testing of Deep Neural Networks Replace Their Online Testing?cited by 24
  4. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 22
  5. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  6. 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17
  7. 2023What Truly Matters in Trajectory Prediction for Autonomous Driving?cited by 17
  8. 2021What data do we need for training an AV motion planner?cited by 15
  9. 2026Fail2Drive: Benchmarking Closed-Loop Driving Generalizationcited by 15
  10. 2021Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settingscited by 14
  11. 2021DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networkscited by 13
  12. 2023CCIL: Context-conditioned imitation learning for urban drivingcited by 12
  13. 2023On Offline Evaluation of 3D Object Detection for Autonomous Drivingcited by 11
  14. 2023Learning to Drive Anywherecited by 9
  15. 2021SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Controlcited by 8
  16. 2025PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Drivingcited by 8
  17. 2021Towards End-to-End Deep Learning for Autonomous Racing: On Data Collection and a Unified Architecture for Steering and Throttle Predictioncited by 7
  18. 2022LiDAR-as-Camera for End-to-End Drivingcited by 7
  19. 2023FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusioncited by 7
  20. 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarkscited by 7
  21. 2021Deep Visual Navigation under Partial Observabilitycited by 6
  22. 2024DualAD: Dual-Layer Planning for Reasoning in Autonomous Drivingcited by 6
  23. 2023Learning with Chemical versus Electrical Synapses -- Does it Make a Difference?cited by 5
  24. 2021Adversarial Mixture Density Networks: Learning to Drive Safely from Collision Datacited by 4
  25. 2024Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systemscited by 4
  26. 2022Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelinescited by 3
  27. 2023DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Datacited by 3
  28. 2023Addressing Limitations of State-Aware Imitation Learning for Autonomous Drivingcited by 3
  29. 2022On the Choice of Data for Efficient Training and Validation of End-to-End Driving Modelscited by 2
  30. 2023Controlling Steering with Energy-Based Modelscited by 2
  31. 2023Combating the effects of speed and delays in end-to-end self-drivingcited by 2
  32. 2021Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cagescited by 1
  33. 2021Self-Supervised Steering Angle Prediction for Vehicle Control Using Visual Odometrycited by 1
  34. 2022Lateral Ego-Vehicle Control without Supervision using Point Cloudscited by 1
  35. 2023Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Informationcited by 1
  36. 2023Privileged to Predicted: Towards Sensorimotor Reinforcement Learning for Urban Drivingcited by 1
  37. 2024Behavioral Cloning Models Reality Check for Autonomous Drivingcited by 1
  38. 2025Simulator Ensembles for Trustworthy Autonomous Driving Systems Testingcited by 1

50 most cited of 62 All 62 (JSON)