# Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey

arXiv: https://arxiv.org/abs/2111.14324 · HTML: https://mlsanity.com/paper/2111.14324/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 147 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 30 of 31)

- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks](https://mlsanity.com/paper/1802.04034/) (2018)
- [Efficient Formal Safety Analysis of Neural Networks](https://mlsanity.com/paper/1809.08098/) (2018)
- [Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning](https://mlsanity.com/paper/1711.06782/) (2017)
- [Safe Exploration in Finite Markov Decision Processes with Gaussian Processes](https://mlsanity.com/paper/1606.04753/) (2016)
- [Safe Reinforcement Learning via Curriculum Induction](https://mlsanity.com/paper/2006.12136/) (2020)
- [Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces](https://mlsanity.com/paper/1902.03229/) (2019)
- [Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences](https://mlsanity.com/paper/2002.09089/) (2020)
- [On Testing Machine Learning Programs](https://mlsanity.com/paper/1812.02257/) (2018)
- [Penalizing side effects using stepwise relative reachability](https://mlsanity.com/paper/1806.01186/) (2018)
- [Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2008.10581/) (2020)
- [Safety Concerns and Mitigation Approaches Regarding the Use of Deep Learning in Safety-Critical Perception Tasks](https://mlsanity.com/paper/2001.08001/) (2020)
- [Towards Safe Policy Improvement for Non-Stationary MDPs](https://mlsanity.com/paper/2010.12645/) (2020)
- [Fast OSCAR and OWL Regression via Safe Screening Rules](https://mlsanity.com/paper/2006.16433/) (2020)
- [Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities](https://mlsanity.com/paper/2008.08221/) (2020)
- [A Safety Framework for Critical Systems Utilising Deep Neural Networks](https://mlsanity.com/paper/2003.05311/) (2020)
- [Efficient, Safe, and Probably Approximately Complete Learning of Action Models](https://mlsanity.com/paper/1705.08961/) (2017)
- [Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret](https://mlsanity.com/paper/1505.05798/) (2015)
- [Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificates](https://mlsanity.com/paper/1912.09957/) (2019)
- [Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Driving](https://mlsanity.com/paper/1903.03438/) (2019)
- [Learning Safe Policies with Expert Guidance](https://mlsanity.com/paper/1805.08313/) (2018)
- [Safe Screening Rules for ℓ0-Regression](https://mlsanity.com/paper/2004.08773/) (2020)
- [Dynamic Risk Assessment for Vehicles of Higher Automation Levels by Deep Learning](https://mlsanity.com/paper/1806.07635/) (2018)
- [Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1704.02882/) (2017)
- [A Machine-learning Based Ensemble Method For Anti-patterns Detection](https://mlsanity.com/paper/1903.01899/) (2019)
- [A Learning Approach to Enhance Assurances for Real-Time Self-Adaptive Systems](https://mlsanity.com/paper/1804.00994/) (2018)
- [A Machine Learning Based Framework for Code Clone Validation](https://mlsanity.com/paper/2005.00967/) (2020)

## Abstract (arXiv)

Machine learning (ML) is finding its way into safety-critical systems (SCS). Current safety standards and practice were not designed to cope with ML techniques, and it is difficult to be confident that SCSs that contain ML components are safe. Our hypothesis was that there has been a rush to deploy ML techniques at the expense of a thorough examination as to whether the use of ML techniques introduces safety problems that we are not yet adequately able to detect and mitigate against. We thus conducted a targeted literature survey to determine the research effort that has been expended in applying ML to SCS compared with that spent on evaluating the safety of SCSs that deploy ML components. This paper presents the (surprising) results of the survey.