# Understanding and Avoiding AI Failures: A Practical Guide

arXiv: https://arxiv.org/abs/2104.12582 · HTML: https://mlsanity.com/paper/2104.12582/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 72 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [AI Safety Gridworlds](https://mlsanity.com/paper/1711.09883/) (2017)
- [The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities](https://mlsanity.com/paper/1803.03453/) (2018)
- [Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure](https://mlsanity.com/paper/1809.07424/) (2018)
- [Safe Exploration of State and Action Spaces in Reinforcement Learning](https://mlsanity.com/paper/1402.0560/) (2014)
- [Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari](https://mlsanity.com/paper/1802.08842/) (2018)
- [Global AI Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence](https://mlsanity.com/paper/1907.07892/) (2019)
- [Morality, Machines and the Interpretation Problem: A Value-based, Wittgensteinian Approach to Building Moral Agents](https://mlsanity.com/paper/2103.02728/) (2021)
- [Safe Artificial General Intelligence via Distributed Ledger Technology](https://mlsanity.com/paper/1902.03689/) (2019)
- [Classification Schemas for Artificial Intelligence Failures](https://mlsanity.com/paper/1907.07771/) (2019)
- [Explainable AI for System Failures: Generating Explanations that Improve Human Assistance in Fault Recovery](https://mlsanity.com/paper/2011.09407/) (2020)

## Built on by (most cited first, 6 of 6)

- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Establishing Meta-Decision-Making for AI: An Ontology of Relevance, Representation and Reasoning](https://mlsanity.com/paper/2210.00608/) (2022)
- [AI Risk Skepticism](https://mlsanity.com/paper/2105.02704/) (2021)
- [Reporting Risks in AI-based Assistive Technology Research: A Systematic Review](https://mlsanity.com/paper/2407.12035/) (2024)
- [The Stories We Govern By: AI, Risk, and the Power of Imaginaries](https://mlsanity.com/paper/2508.11729/) (2025)

## Abstract (arXiv)

As AI technologies increase in capability and ubiquity, AI accidents are becoming more common. Based on normal accident theory, high reliability theory, and open systems theory, we create a framework for understanding the risks associated with AI applications. In addition, we also use AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI. Together, these two fields give a more complete picture of the risks of contemporary AI. By focusing on system properties near accidents instead of seeking a root cause of accidents, we identify where attention should be paid to safety for current generation AI systems.