# Maturity Framework for Enhancing Machine Learning Quality

arXiv: https://arxiv.org/abs/2502.15758 · HTML: https://mlsanity.com/paper/2502.15758/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 79 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)
- [Machine Learning Operations (MLOps): Overview, Definition, and Architecture](https://mlsanity.com/paper/2205.02302/) (2022)
- [Principles and Practice of Explainable Machine Learning](https://mlsanity.com/paper/2009.11698/) (2020)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning](https://mlsanity.com/paper/2007.09339/) (2020)
- [Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology](https://mlsanity.com/paper/2003.05155/) (2020)
- [Adoption and Effects of Software Engineering Best Practices in Machine Learning](https://mlsanity.com/paper/2007.14130/) (2020)
- [A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges](https://mlsanity.com/paper/2012.07919/) (2020)
- [E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling](https://mlsanity.com/paper/2108.13298/) (2021)
- [Efficient Machine Learning for Big Data: A Review](https://mlsanity.com/paper/1503.05296/) (2015)
- [Characterizing machine learning process: A maturity framework](https://mlsanity.com/paper/1811.04871/) (2018)
- [MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoid](https://mlsanity.com/paper/2211.05232/) (2022)
- [Deepchecks: A Library for Testing and Validating Machine Learning Models and Data](https://mlsanity.com/paper/2203.08491/) (2022)
- [Democratizing online controlled experiments at Booking.com](https://mlsanity.com/paper/1710.08217/) (2017)
- [Quality Management of Machine Learning Systems](https://mlsanity.com/paper/2006.09529/) (2020)
- [Text2Topic: Multi-Label Text Classification System for Efficient Topic Detection in User Generated Content with Zero-Shot Capabilities](https://mlsanity.com/paper/2310.14817/) (2023)
- [Impact of Response Latency on User Behaviour in Mobile Web Search](https://mlsanity.com/paper/2101.09086/) (2021)
- [Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phases](https://mlsanity.com/paper/2105.01195/) (2021)
- [A Review of Software Quality Models for the Evaluation of Software Products](https://mlsanity.com/paper/1412.2977/) (2014)
- [Mining the Stars: Learning Quality Ratings with User-facing Explanations for Vacation Rentals](https://mlsanity.com/paper/2101.10737/) (2021)
- [With One Voice: Composing a Travel Voice Assistant from Re-purposed Models](https://mlsanity.com/paper/2108.11463/) (2021)

## Abstract (arXiv)

With the rapid integration of Machine Learning (ML) in business applications and processes, it is crucial to ensure the quality, reliability and reproducibility of such systems. We suggest a methodical approach towards ML system quality assessment and introduce a structured Maturity framework for governance of ML. We emphasize the importance of quality in ML and the need for rigorous assessment, driven by issues in ML governance and gaps in existing frameworks. Our primary contribution is a comprehensive open-sourced quality assessment method, validated with empirical evidence, accompanied by a systematic maturity framework tailored to ML systems. Drawing from applied experience at Booking.com, we discuss challenges and lessons learned during large-scale adoption within organizations. The study presents empirical findings, highlighting quality improvement trends and showcasing business outcomes. The maturity framework for ML systems, aims to become a valuable resource to reshape industry standards and enable a structural approach to improve ML maturity in any organization.