# Naming the Pain in Machine Learning-Enabled Systems Engineering

arXiv: https://arxiv.org/abs/2406.04359 · HTML: https://mlsanity.com/paper/2406.04359/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 11 of 11)

- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Requirements Engineering for Machine Learning: Perspectives from Data Scientists](https://mlsanity.com/paper/1908.04674/) (2019)
- [Software Engineering Challenges of Deep Learning](https://mlsanity.com/paper/1810.12034/) (2018)
- [A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges](https://mlsanity.com/paper/2012.07919/) (2020)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey](https://mlsanity.com/paper/2310.06726/) (2023)
- [What Makes Agile Software Development Agile?](https://mlsanity.com/paper/2109.11435/) (2021)
- [Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach](https://mlsanity.com/paper/2309.07980/) (2023)
- [ML-Enabled Systems Model Deployment and Monitoring: Status Quo and Problems](https://mlsanity.com/paper/2402.05333/) (2024)
- [Challenges in Survey Research](https://mlsanity.com/paper/1908.05899/) (2019)

## Built on by (most cited first, 3 of 3)

- [Towards Requirements Engineering for RAG Systems](https://mlsanity.com/paper/2505.07553/) (2025)
- [Define-ML: An Approach to Ideate Machine Learning-Enabled Systems](https://mlsanity.com/paper/2506.20621/) (2025)
- [Exploring CoCo Challenges in ML Engineering Teams: Insights From the Semiconductor Industry](https://mlsanity.com/paper/2605.07389/) (2026)

## Abstract (arXiv)

Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems.