Safety Concerns and Mitigation Approaches Regarding the Use of Deep Learning in Safety-Critical Perception Tasks
Not on the map. Why?
Abstract (arXiv)
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning approaches typically exhibit a black-box behavior which makes it hard for them to be evaluated with respect to safety-critical aspects. While there have been some work on safety in deep learning, most papers typically focus on high-level safety concerns. In this work, we seek to dive into the safety concerns of deep learning methods and present a concise enumeration on a deeply technical level. Additionally, we present extensive discussions on possible mitigation methods and give an outlook regarding what mitigation methods are still missing in order to facilitate an argumentation for the safety of a deep learning method.
Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jan 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 6; 2023: 5; 2024: 7; 2025: 3; 2026: 2.
31 papers.
Sort:
- 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 52
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
- 2023Labeling Neural Representations with Inverse Recognitioncited by 31
- 2021Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detectioncited by 23
- 2022Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake Systemcited by 14
- 2021Exposing Previously Undetectable Faults in Deep Neural Networkscited by 11
- 2023Deep Learning Safety Concerns in Automated Driving Perceptioncited by 10
- 2021A Review of Testing Object-Based Environment Perception for Safe Automated Drivingcited by 9
- 2022Combating noisy labels in object detection datasetscited by 7
- 2021Requirement Engineering Challenges for AI-intense Systems Developmentcited by 6
- 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Studycited by 5
- 2021Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phasescited by 4
19 more
- 2022Tailored Uncertainty Estimation for Deep Learning Systemscited by 3
- 2022Verifiable Obstacle Detectioncited by 3
- 2023SURE-Val: Safe Urban Relevance Extension and Validationcited by 3
- 2023AI Hazard Management: A framework for the systematic management of root causes for AI riskscited by 3
- 2022Mitigating Shadows in Lidar Scan Matching using Spherical Voxelscited by 2
- 2024A Safety-Adapted Loss for Pedestrian Detection in Automated Drivingcited by 2
- 2025Runtime Safety Monitoring of Deep Neural Networks for Perception: A Surveycited by 2
- 2021Single-Step Adversarial Training for Semantic Segmentationcited by 1
- 2022Perception Simplex: Verifiable Collision Avoidance in Autonomous Vehicles Amidst Obstacle Detection Faultscited by 1
- 2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirementcited by 1
- 2021Is the Rush to Machine Learning Jeopardizing Safety? Results of a Surveycited by 0
- 2023Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Drivingcited by 0
- 2024Learning-Based Error Detection System for Advanced Vehicle Instrument Cluster Renderingcited by 0
- 2024Methodology for an Analysis of Influencing Factors on 3D Object Detection Performancecited by 0
- 2024Landscape of AI safety concerns -- A methodology to support safety assurance for AI-based autonomous systemscited by 0
- 2025A Systematic Literature Review on Safety of the Intended Functionality for Automated Driving Systemscited by 0
- 2025A Machine Learning Perspective on Automated Driving Corner Casescited by 0
- 2026Towards a Systematic Risk Assessment of Deep Neural Network Limitations in Autonomous Driving Perceptioncited by 0
- 2026From Legal Text to AI-specific Risk Sources: A Systematic Analysis of the EU AI Act's High-Risk Requirementscited by 0
- 2026From Legal Text to AI-specific Risk Sources: A Systematic Analysis of the EU AI Act's High-Risk Requirements11 Sep 2026
- 2026Towards a Systematic Risk Assessment of Deep Neural Network Limitations in Autonomous Driving Perception21 Apr 2026
- 2025Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey8 Nov 2025
- 2025A Machine Learning Perspective on Automated Driving Corner Cases12 Oct 2025
- 2025A Systematic Literature Review on Safety of the Intended Functionality for Automated Driving Systems4 Mar 2025
- 2024Landscape of AI safety concerns -- A methodology to support safety assurance for AI-based autonomous systems18 Dec 2024
- 2024Methodology for an Analysis of Influencing Factors on 3D Object Detection Performance13 Nov 2024
- 2024Learning-Based Error Detection System for Advanced Vehicle Instrument Cluster Rendering4 Sep 2024
- 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study31 May 2024
- 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review8 Feb 2024
- 2024A Safety-Adapted Loss for Pedestrian Detection in Automated Driving5 Feb 2024
- 2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirement18 Jan 2024
19 more
- 2023Labeling Neural Representations with Inverse Recognition22 Nov 2023
- 2023AI Hazard Management: A framework for the systematic management of root causes for AI risks25 Oct 2023
- 2023Deep Learning Safety Concerns in Automated Driving Perception7 Sep 2023
- 2023Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Driving14 Aug 2023
- 2023SURE-Val: Safe Urban Relevance Extension and Validation4 Aug 2023
- 2022Combating noisy labels in object detection datasets25 Nov 2022
- 2022Perception Simplex: Verifiable Collision Avoidance in Autonomous Vehicles Amidst Obstacle Detection Faults4 Sep 2022
- 2022Verifiable Obstacle Detection30 Aug 2022
- 2022Mitigating Shadows in Lidar Scan Matching using Spherical Voxels1 Aug 2022
- 2022Tailored Uncertainty Estimation for Deep Learning Systems29 Apr 2022
- 2022Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System16 Apr 2022
- 2021Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey29 Nov 2021
- 2021Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection18 Oct 2021
- 2021Single-Step Adversarial Training for Semantic Segmentation30 Jun 2021
- 2021Exposing Previously Undetectable Faults in Deep Neural Networks1 Jun 2021
- 2021Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phases3 May 2021
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety29 Apr 2021
- 2021Requirement Engineering Challenges for AI-intense Systems Development18 Mar 2021
- 2021A Review of Testing Object-Based Environment Perception for Safe Automated Driving16 Feb 2021
- 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 52
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
- 2023Labeling Neural Representations with Inverse Recognitioncited by 31
- 2021Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detectioncited by 23
- 2022Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake Systemcited by 14
- 2021Exposing Previously Undetectable Faults in Deep Neural Networkscited by 11
- 2023Deep Learning Safety Concerns in Automated Driving Perceptioncited by 10
- 2021A Review of Testing Object-Based Environment Perception for Safe Automated Drivingcited by 9
- 2022Combating noisy labels in object detection datasetscited by 7
- 2021Requirement Engineering Challenges for AI-intense Systems Developmentcited by 6
- 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Studycited by 5
- 2021Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phasescited by 4
12 most cited of 31 All 31 (JSON)
- 2026From Legal Text to AI-specific Risk Sources: A Systematic Analysis of the EU AI Act's High-Risk Requirements11 Sep 2026
- 2026Towards a Systematic Risk Assessment of Deep Neural Network Limitations in Autonomous Driving Perception21 Apr 2026
- 2025Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey8 Nov 2025
- 2025A Machine Learning Perspective on Automated Driving Corner Cases12 Oct 2025
- 2025A Systematic Literature Review on Safety of the Intended Functionality for Automated Driving Systems4 Mar 2025
- 2024Landscape of AI safety concerns -- A methodology to support safety assurance for AI-based autonomous systems18 Dec 2024
- 2024Methodology for an Analysis of Influencing Factors on 3D Object Detection Performance13 Nov 2024
- 2024Learning-Based Error Detection System for Advanced Vehicle Instrument Cluster Rendering4 Sep 2024
- 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study31 May 2024
- 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review8 Feb 2024
- 2024A Safety-Adapted Loss for Pedestrian Detection in Automated Driving5 Feb 2024
- 2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirement18 Jan 2024
12 newest of 31 All 31 (JSON)