mlsanityData to 14 Oct 2026

Safety Monitoring of Machine Learning Perception Functions: a Survey

arXiv 2412.06869 ↗First posted 9 Dec 2024cs.LG cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Machine Learning (ML) models, such as deep neural networks, are widely applied in autonomous systems to perform complex perception tasks. New dependability challenges arise when ML predictions are used in safety-critical applications, like autonomous cars and surgical robots. Thus, the use of fault tolerance mechanisms, such as safety monitors, is essential to ensure the safe behavior of the system despite the occurrence of faults. This paper presents an extensive literature review on safety monitoring of perception functions using ML in a safety-critical context. In this review, we structure the existing literature to highlight key factors to consider when designing such monitors: threat identification, requirements elicitation, detection of failure, reaction, and evaluation. We also highlight the ongoing challenges associated with safety monitoring and suggest directions for future research.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 68 of its 159 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

68 of 159 references.

References · 68

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  5. 2016Adversarial examples in the physical worldcited by 1,769
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  8. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  10. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  11. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  12. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
38 more
  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  2. 2019Three scenarios for continual learningcited by 614
  3. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  4. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  5. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  6. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  7. 2019Deep Learning for Image Super-resolution: A Surveycited by 369
  8. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  9. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 270
  10. 2018Adversarial Attacks and Defences: A Surveycited by 259
  11. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  12. 2017Concrete Dropoutcited by 227
  13. 2018Adversarially Learned One-Class Classifier for Novelty Detectioncited by 202
  14. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  15. 2019Deep Learning on Image Denoising: An overviewcited by 128
  16. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  17. 2021Machine Learning with a Reject Option: A surveycited by 109
  18. 2017Logic Tensor Networks for Semantic Image Interpretationcited by 92
  19. 2018Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distancecited by 74
  20. 2021No One Representation to Rule Them All: Overlapping Features of Training Methodscited by 58
  21. 2019Misbehaviour Prediction for Autonomous Driving Systemscited by 55
  22. 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 53
  23. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  24. 2018Dirichlet-based Gaussian Processes for Large-scale Calibrated Classificationcited by 51
  25. 2018Adversarial Sample Detection for Deep Neural Network through Model Mutation Testingcited by 48
  26. 2018Efficient Uncertainty Estimation for Semantic Segmentation in Videoscited by 46
  27. 2020Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systemscited by 46
  28. 2018Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industrycited by 38
  29. 2019Outside the Box: Abstraction-Based Monitoring of Neural Networkscited by 36
  30. 2017Compositional Falsification of Cyber-Physical Systems with Machine Learning Componentscited by 32
  31. 2019Attacking Vision-based Perception in End-to-End Autonomous Driving Modelscited by 31
  32. 2019Neural Simplex Architecturecited by 29
  33. 2020Bayesian Image Reconstruction using Deep Generative Modelscited by 29
  34. 2019Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Studycited by 27
  35. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 25
  36. 2020Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunitiescited by 23
  37. 2022Out-Of-Distribution Detection Is Not All You Needcited by 23
  38. 2020Model Assertions for Monitoring and Improving ML Modelscited by 22

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Runtime Safety Monitoring of Deep Neural Networks for Perception: A Surveycited by 2
  2. 2026Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objectscited by 0
  3. 2026Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landingcited by 0