# Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database

arXiv: https://arxiv.org/abs/2011.08512 · HTML: https://mlsanity.com/paper/2011.08512/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 30 of 96)

- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- ["Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice](https://mlsanity.com/paper/2212.14315/) (2022)
- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Neuro-Symbolic Verification of Deep Neural Networks](https://mlsanity.com/paper/2203.00938/) (2022)
- [A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms](https://mlsanity.com/paper/2407.01294/) (2024)
- [Pixel-Grounded Prototypical Part Networks](https://mlsanity.com/paper/2309.14531/) (2023)
- [Not My Voice! A Taxonomy of Ethical and Safety Harms of Speech Generators](https://mlsanity.com/paper/2402.01708/) (2024)
- [The AI Agent Index](https://mlsanity.com/paper/2502.01635/) (2025)
- [AI Ethics Issues in Real World: Evidence from AI Incident Database](https://mlsanity.com/paper/2206.07635/) (2022)
- [Robots Enact Malignant Stereotypes](https://mlsanity.com/paper/2207.11569/) (2022)
- [Unfooling Perturbation-Based Post Hoc Explainers](https://mlsanity.com/paper/2205.14772/) (2022)
- [fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks](https://mlsanity.com/paper/2302.11703/) (2023)
- [Responsible Reporting for Frontier AI Development](https://mlsanity.com/paper/2404.02675/) (2024)
- [Threat Assessment in Machine Learning based Systems](https://mlsanity.com/paper/2207.00091/) (2022)
- [A taxonomic system for failure cause analysis of open source AI incidents](https://mlsanity.com/paper/2211.07280/) (2022)
- [Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie](https://mlsanity.com/paper/2309.09120/) (2023)
- [The AI Incident Database as an Educational Tool to Raise Awareness of AI Harms: A Classroom Exploration of Efficacy, Limitations, & Future Improvements](https://mlsanity.com/paper/2310.06269/) (2023)
- [RealHarm: A Collection of Real-World Language Model Application Failures](https://mlsanity.com/paper/2504.10277/) (2025)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [The Role of Governments in Increasing Interconnected Post-Deployment Monitoring of AI](https://mlsanity.com/paper/2410.04931/) (2024)
- [In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI](https://mlsanity.com/paper/2503.16861/) (2025)
- [BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies](https://mlsanity.com/paper/2405.06783/) (2024)
- [Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models](https://mlsanity.com/paper/2508.16673/) (2025)
- [Lessons for Editors of AI Incidents from the AI Incident Database](https://mlsanity.com/paper/2409.16425/) (2024)
- [To Err is AI : A Case Study Informing LLM Flaw Reporting Practices](https://mlsanity.com/paper/2410.12104/) (2024)
- [Standardised schema and taxonomy for AI incident databases in critical digital infrastructure](https://mlsanity.com/paper/2501.17037/) (2025)
- [Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice](https://mlsanity.com/paper/2506.09873/) (2025)
- [Indexing AI Risks with Incidents, Issues, and Variants](https://mlsanity.com/paper/2211.10384/) (2022)

## Abstract (arXiv)

Mature industrial sectors (e.g., aviation) collect their real world failures in incident databases to inform safety improvements. Intelligent systems currently cause real world harms without a collective memory of their failings. As a result, companies repeatedly make the same mistakes in the design, development, and deployment of intelligent systems. A collection of intelligent system failures experienced in the real world (i.e., incidents) is needed to ensure intelligent systems benefit people and society. The AI Incident Database is an incident collection initiated by an industrial/non-profit cooperative to enable AI incident avoidance and mitigation. The database supports a variety of research and development use cases with faceted and full text search on more than 1,000 incident reports archived to date.