# AI Assurance: A Comprehensive Testing Strategy for Enterprise AI Systems

arXiv: https://arxiv.org/abs/2605.23459 · HTML: https://mlsanity.com/paper/2605.23459/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 10 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 1 of 1)

- [Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecture](https://mlsanity.com/paper/2411.13768/) (2024)

## Built on by (most cited first, 1 of 1)

- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)

## Abstract (arXiv)

Enterprise AI systems, built on large language models, retrieval pipelines and autonomous agents, introduce a class of risks that traditional software quality assurance was never designed to address. These systems are probabilistic, context-sensitive and emergent: they cannot be verified to be correct in the classical sense, but only evaluated with increasing confidence. This paper presents a comprehensive assurance strategy for enterprise AI systems built around three key principles: first, that AI testing should focus on continuous risk reduction rather than strict correctness verification; second, that evaluation must be treated as a core engineering discipline alongside development; and third, that failures in AI assurance can lead to organizational impacts that are fundamentally different from those seen in traditional deterministic software systems. We introduce a structured AI Failure Taxonomy, propose a revised five-layer AI Assurance Pyramid and provide operational guidance on evaluation-driven development, RAG system testing, model lifecycle management and governance. The goal is to equip engineering leaders and practitioners with a strategy that is both philosophically grounded and operationally deployable.