# Quality Management of Machine Learning Systems

arXiv: https://arxiv.org/abs/2006.09529 · HTML: https://mlsanity.com/paper/2006.09529/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 6 of 6)

- [Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2103.09783/) (2021)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [What is Software Quality for AI Engineers? Towards a Thinning of the Fog](https://mlsanity.com/paper/2203.12697/) (2022)
- [Maturity Framework for Enhancing Machine Learning Quality](https://mlsanity.com/paper/2502.15758/) (2025)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)
- [Reconsidering Requirements Engineering: Human-AI Collaboration in AI-Native Software Development](https://mlsanity.com/paper/2510.04380/) (2025)

## Abstract (arXiv)

In the past decade, Artificial Intelligence (AI) has become a part of our daily lives due to major advances in Machine Learning (ML) techniques. In spite of an explosive growth in the raw AI technology and in consumer facing applications on the internet, its adoption in business applications has conspicuously lagged behind. For business/mission-critical systems, serious concerns about reliability and maintainability of AI applications remain. Due to the statistical nature of the output, software 'defects' are not well defined. Consequently, many traditional quality management techniques such as program debugging, static code analysis, functional testing, etc. have to be reevaluated. Beyond the correctness of an AI model, many other new quality attributes, such as fairness, robustness, explainability, transparency, etc. become important in delivering an AI system. The purpose of this paper is to present a view of a holistic quality management framework for ML applications based on the current advances and identify new areas of software engineering research to achieve a more trustworthy AI.