# Quality Assurance for AI-based Systems: Overview and Challenges

arXiv: https://arxiv.org/abs/2102.05351 · HTML: https://mlsanity.com/paper/2102.05351/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2021.

## Built on by (most cited first, 19 of 19)

- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2103.09783/) (2021)
- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice](https://mlsanity.com/paper/2301.09001/) (2023)
- [Interactive Machine Learning: A State of the Art Review](https://mlsanity.com/paper/2207.06196/) (2022)
- [Towards a Common Testing Terminology for Software Engineering and Data Science Experts](https://mlsanity.com/paper/2108.13837/) (2021)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phases](https://mlsanity.com/paper/2105.01195/) (2021)
- [A Survey of Neural Network Robustness Assessment in Image Recognition](https://mlsanity.com/paper/2404.08285/) (2024)
- [Assessing the Auditability of AI-integrating Systems: A Framework and Learning Analytics Case Study](https://mlsanity.com/paper/2411.08906/) (2024)
- [What is Software Quality for AI Engineers? Towards a Thinning of the Fog](https://mlsanity.com/paper/2203.12697/) (2022)
- [Computer Vision Intelligence Test Modeling and Generation: A Case Study on Smart OCR](https://mlsanity.com/paper/2410.03536/) (2024)
- [Awareness requirement and performance management for adaptive systems: a survey](https://mlsanity.com/paper/2302.05518/) (2023)
- [Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks](https://mlsanity.com/paper/2401.12261/) (2024)
- [Towards Sustainability Model Cards](https://mlsanity.com/paper/2507.19559/) (2025)
- [AgentSLA : Towards a Service Level Agreement for AI Agents](https://mlsanity.com/paper/2511.02885/) (2025)
- [The Impact of Environment Configurations on the Stability of AI-Enabled Systems](https://mlsanity.com/paper/2408.02825/) (2024)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)
- [Reconsidering Requirements Engineering: Human-AI Collaboration in AI-Native Software Development](https://mlsanity.com/paper/2510.04380/) (2025)

## Abstract (arXiv)

The number and importance of AI-based systems in all domains is growing. With the pervasive use and the dependence on AI-based systems, the quality of these systems becomes essential for their practical usage. However, quality assurance for AI-based systems is an emerging area that has not been well explored and requires collaboration between the SE and AI research communities. This paper discusses terminology and challenges on quality assurance for AI-based systems to set a baseline for that purpose. Therefore, we define basic concepts and characterize AI-based systems along the three dimensions of artifact type, process, and quality characteristics. Furthermore, we elaborate on the key challenges of (1) understandability and interpretability of AI models, (2) lack of specifications and defined requirements, (3) need for validation data and test input generation, (4) defining expected outcomes as test oracles, (5) accuracy and correctness measures, (6) non-functional properties of AI-based systems, (7) self-adaptive and self-learning characteristics, and (8) dynamic and frequently changing environments.