# Taxonomy of Machine Learning Safety: A Survey and Primer

arXiv: https://arxiv.org/abs/2106.04823 · HTML: https://mlsanity.com/paper/2106.04823/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 219 of its 306 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 30 of 219)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)

## Built on by (most cited first, 25 of 25)

- [Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation](https://mlsanity.com/paper/2305.02231/) (2023)
- [Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy](https://mlsanity.com/paper/2402.04247/) (2024)
- [TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories](https://mlsanity.com/paper/2511.21322/) (2025)
- [Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasets](https://mlsanity.com/paper/2401.17013/) (2024)
- [System Safety Monitoring of Learned Components Using Temporal Metric Forecasting](https://mlsanity.com/paper/2405.13254/) (2024)
- [You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks](https://mlsanity.com/paper/2306.09951/) (2023)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models](https://mlsanity.com/paper/2501.18463/) (2025)
- [Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems](https://mlsanity.com/paper/2510.21254/) (2025)
- [Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey](https://mlsanity.com/paper/2511.05982/) (2025)
- [Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift](https://mlsanity.com/paper/2603.01040/) (2026)
- [The Chai Platform's AI Safety Framework](https://mlsanity.com/paper/2306.02979/) (2023)
- [Machine Learning Operations: A Mapping Study](https://mlsanity.com/paper/2409.19416/) (2024)
- [SoK: Towards Security and Safety of Edge AI](https://mlsanity.com/paper/2410.05349/) (2024)
- [Revisiting Energy-Based Model for Out-of-Distribution Detection](https://mlsanity.com/paper/2412.03058/) (2024)
- [Detecting underdetermination in parameterized quantum circuits](https://mlsanity.com/paper/2504.03315/) (2025)
- [Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction](https://mlsanity.com/paper/2510.15422/) (2025)
- [Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward](https://mlsanity.com/paper/2601.13122/) (2026)
- [Synergistic Simplex: Cooperative Runtime Assurance for Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2605.08190/) (2026)
- [AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines](https://mlsanity.com/paper/2605.10987/) (2026)
- [A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning](https://mlsanity.com/paper/2606.16933/) (2026)
- [A Deterministic and Auditable AI Security Risk Assessment Framework with ATLAS Aligned Executable Rules and Formal Verification](https://mlsanity.com/paper/2610.01436/) (2026)
- [Dependable AI-Assisted Engineering: A Formal Framework for AI Participation and Assurance in Safety-Critical Workflows](https://mlsanity.com/paper/2610.04084/) (2026)
- [Actionable Meta-Requirements for AI Robustness Auditing](https://mlsanity.com/paper/2610.14356/) (2026)
- [Learning Adaptive Deliberation Control for Reliable Recognition with Multi-Agent Language Models](https://mlsanity.com/paper/2610.17840/) (2026)

## Abstract (arXiv)

The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations. Research explores different approaches to improve ML dependability by proposing new models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. However, there is a missing connection between ongoing ML research and well-established safety principles. In this paper, we present a structured and comprehensive review of ML techniques to improve the dependability of ML algorithms in uncontrolled open-world settings. From this review, we propose the Taxonomy of ML Safety that maps state-of-the-art ML techniques to key engineering safety strategies. Our taxonomy of ML safety presents a safety-oriented categorization of ML techniques to provide guidance for improving dependability of the ML design and development. The proposed taxonomy can serve as a safety checklist to aid designers in improving coverage and diversity of safety strategies employed in any given ML system.