mlsanityData to 14 Oct 2026

Correct-by-Construction Runtime Enforcement in AI -- A Survey

arXiv 2208.14426 ↗First posted 30 Aug 2022cs.AI cs.LOcited by 12

Not on the map. Why?

Abstract (arXiv)

Runtime enforcement refers to the theories, techniques, and tools for enforcing correct behavior with respect to a formal specification of systems at runtime. In this paper, we are interested in techniques for constructing runtime enforcers for the concrete application domain of enforcing safety in AI. We discuss how safety is traditionally handled in the field of AI and how more formal guarantees on the safety of a self-learning agent can be given by integrating a runtime enforcer. We survey a selection of work on such enforcers, where we distinguish between approaches for discrete and continuous action spaces. The purpose of this paper is to foster a better understanding of advantages and limitations of different enforcement techniques, focusing on the specific challenges that arise due to their application in AI. Finally, we present some open challenges and avenues for future work.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 43 of its 71 references that are in the map. First posted on arXiv 30 Aug 2022.

Builds on

43 of 71 references.

References · 43

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  8. 2017Constrained Policy Optimizationcited by 983
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  10. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  11. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
  12. 2017Safe Reinforcement Learning via Shieldingcited by 449
31 more
  1. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  2. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  3. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  4. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  6. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  7. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  8. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  9. 2016Grad-CAM: Why did you say that?cited by 161
  10. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  11. 2016Learning in Implicit Generative Modelscited by 125
  12. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
  13. 2020Cautious Reinforcement Learning with Logical Constraintscited by 45
  14. 2015Shield Synthesis: Runtime Enforcement for Reactive Systemscited by 42
  15. 2020Out-of-Distribution Detection for Automotive Perceptioncited by 37
  16. 2016Towards Verified Artificial Intelligencecited by 36
  17. 2019Certified Reinforcement Learning with Logic Guidancecited by 36
  18. 2019Outside the Box: Abstraction-Based Monitoring of Neural Networkscited by 36
  19. 2019Autonomous Highway Driving using Deep Reinforcement Learningcited by 22
  20. 2020Into the Unknown: Active Monitoring of Neural Networkscited by 19
  21. 2015Universum Prescription: Regularization using Unlabeled Datacited by 16
  22. 2018Counterexample-Guided Data Augmentationcited by 15
  23. 2019Practical Solutions for Machine Learning Safety in Autonomous Vehiclescited by 15
  24. 2021Shielding Atari Games with Bounded Presciencecited by 14
  25. 2020Online Shielding for Stochastic Systemscited by 11
  26. 2014An Extensible Benchmarking Infrastructure for Motion Planning Algorithmscited by 9
  27. 2018A Survey of Challenges for Runtime Verification from Advanced Application Domains (Beyond Software)cited by 9
  28. 2020Data-efficient visuomotor policy training using reinforcement learning and generative modelscited by 5
  29. 2019Risk-Aware Planning by Confidence Estimation using Deep Learning-Based Perceptioncited by 2
  30. 2021Interpretability in Contact-Rich Manipulation via Kinodynamic Imagescited by 1
  31. 2021Learning Safe and Optimal Control Strategies for Storm Water Detention Pondscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 2; 2026: 4.

12 papers.

  1. 2024Compositional Shielding and Reinforcement Learning for Multi-Agent Systemscited by 8
  2. 2024Provably Safe Neural Network Controllers via Differential Dynamic Logiccited by 7
  3. 2024CommonPower: A Framework for Safe Data-Driven Smart Grid Controlcited by 4
  4. 2023Safety Shielding under Delayed Observationcited by 3
  5. 2024Composing Reinforcement Learning Policies, with Formal Guaranteescited by 2
  6. 2025Verification of Autonomous Neural Car Control with KeYmaera Xcited by 1
  7. 2025Follow the STARs: Dynamic ω-Regular Shielding of Learned Policiescited by 1
  8. 2023Experiments with Detecting and Mitigating AI Deceptioncited by 0
  9. 2026Explainably Safe Reinforcement Learningcited by 0
  10. 2026Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agentscited by 0
  11. 2026MENTOR: Procedural Shielding of Tool-Using LLM Agentscited by 0
  12. 2026One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AIcited by 0