mlsanityData to 13 Oct 2026

Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry

arXiv 1812.05389 ↗First posted 13 Dec 2018cs.SEcited by 38

Not on the map. Why?

Abstract (arXiv)

Deep Neural Networks (DNN) will emerge as a cornerstone in automotive software engineering. However, developing systems with DNNs introduces novel challenges for safety assessments. This paper reviews the state-of-the-art in verification and validation of safety-critical systems that rely on machine learning. Furthermore, we report from a workshop series on DNNs for perception with automotive experts in Sweden, confirming that ISO 26262 largely contravenes the nature of DNNs. We recommend aerospace-to-automotive knowledge transfer and systems-based safety approaches, e.g., safety cage architectures and simulated system test cases.

Key facts. Cited by 38 papers in the map (data through 13 Oct 2026). First posted on arXiv 13 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 13; 2022: 9; 2023: 7; 2024: 6; 2025: 1; 2026: 2.

38 papers.

Sort:

By area:

  1. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  2. 2021Software Engineering for AI-Based Systems: A Surveycited by 68
  3. 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
  4. 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
  5. 2022Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake Systemcited by 14
  6. 2021Increasing the Confidence of Deep Neural Networks by Coverage Analysiscited by 9
  7. 2021A Review of Testing Object-Based Environment Perception for Safe Automated Drivingcited by 9
  8. 2021Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networkscited by 9
  9. 2021Efficient and Effective Generation of Test Cases for Pedestrian Detection -- Search-based Software Testing of Baidu Apollo in SVLcited by 8
  10. 2021Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutionscited by 7
  11. 2022Performance Analysis of Out-of-Distribution Detection on Trained Neural Networkscited by 7
  12. 2022Requirements Engineering for Machine Learning: A Review and Reflectioncited by 7
26 more
  1. 2021Exploring the Assessment List for Trustworthy AI in the Context of Advanced Driver-Assistance Systemscited by 6
  2. 2021Requirement Engineering Challenges for AI-intense Systems Developmentcited by 6
  3. 2022Maximum Likelihood Uncertainty Estimation: Robustness to Outlierscited by 6
  4. 2022Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applicationscited by 6
  5. 2024Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasetscited by 5
  6. 2024Artificial Intelligence in Industry 4.0: A Review of Integration Challenges for Industrial Systemscited by 5
  7. 2021Agility in Software 2.0 -- Notebook Interfaces and MLOps with Buttresses and Rebarscited by 4
  8. 2022Machine Learning Testing in an ADAS Case Study Using Simulation-Integrated Bio-Inspired Search-Based Testingcited by 4
  9. 2023An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applicationscited by 4
  10. 2023Automotive Perception Software Development: An Empirical Investigation into Data, Annotation, and Ecosystem Challengescited by 4
  11. 2021Test Automation with Grad-CAM Heatmaps -- A Future Pipe Segment in MLOps for Vision AI?cited by 3
  12. 2022Exploring ML testing in practice -- Lessons learned from an interactive rapid review with Axis Communicationscited by 3
  13. 2024Safety Monitoring of Machine Learning Perception Functions: a Surveycited by 3
  14. 2023MLTEing Models: Negotiating, Evaluating, and Documenting Model and System Qualitiescited by 2
  15. 2024Reward Bound for Behavioral Guarantee of Model-based Planning Agentscited by 2
  16. 2024Forward Learning for Gradient-based Black-box Saliency Map Generationcited by 2
  17. 2024Operational range bounding of spectroscopy models with anomaly detectioncited by 2
  18. 2021Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cagescited by 1
  19. 2022Understanding the Impact of Edge Cases from Occluded Pedestrians for ML Systemscited by 1
  20. 2023AI for Agile development: a Meta-Analysiscited by 1
  21. 2023Trustworthy, responsible, ethical AI in manufacturing and supply chains: synthesis and emerging research questionscited by 1
  22. 2023InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogatescited by 1
  23. 2025An LLM-Empowered Adaptive Evolutionary Algorithm For Multi-Component Deep Learning Systemscited by 1
  24. 2022Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practicecited by 0
  25. 2026Data Leakage in Automotive Perception: Practitioners' Insightscited by 0
  26. 2026Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Modelscited by 0