mlsanityData to 14 Oct 2026

A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners

arXiv 2304.00078 ↗First posted 31 Mar 2023cs.SE cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Incorporating machine learning (ML) components into software products raises new software-engineering challenges and exacerbates existing challenges. Many researchers have invested significant effort in understanding the challenges of industry practitioners working on building products with ML components, through interviews and surveys with practitioners. With the intention to aggregate and present their collective findings, we conduct a meta-summary study: We collect 50 relevant papers that together interacted with over 4758 practitioners using guidelines for systematic literature reviews. We then collected, grouped, and organized the over 500 mentions of challenges within those papers. We highlight the most commonly reported challenges and hope this meta-summary will be a useful resource for the research community to prioritize research and education in this field.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 45 of its 140 references that are in the map. First posted on arXiv 31 Mar 2023.

Builds on

45 of 140 references.

References · 45

  1. 2018Model Cards for Model Reportingcited by 1,080
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  3. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  4. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 218
  5. 2019Explainable Machine Learning in Deploymentcited by 211
  6. 2020Challenges in Deploying Machine Learning: a Survey of Case Studiescited by 152
  7. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 135
  8. 2020Adversarial Machine Learning -- Industry Perspectivescited by 122
  9. 2019Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AIcited by 79
  10. 2021Software Engineering for AI-Based Systems: A Surveycited by 69
  11. 2020How do Data Science Workers Collaborate? Roles, Workflows, and Toolscited by 64
  12. 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
33 more
  1. 2019Requirements Engineering for Machine Learning: Perspectives from Data Scientistscited by 49
  2. 2018On Testing Machine Learning Programscited by 48
  3. 2018Software Engineering Challenges of Deep Learningcited by 41
  4. 2020Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodologycited by 41
  5. 2019Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challengescited by 39
  6. 2018Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industrycited by 38
  7. 2021Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?cited by 37
  8. 2020Adoption and Effects of Software Engineering Best Practices in Machine Learningcited by 33
  9. 2020AI Lifecycle Models Need To Be Revised. An Exploratory Study in Fintechcited by 33
  10. 2020A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challengescited by 33
  11. 2021How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Studycited by 29
  12. 2022From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible MLcited by 27
  13. 2017An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Softwarecited by 26
  14. 2022Operationalizing Machine Learning: An Interview Studycited by 25
  15. 2019Studying Software Engineering Patterns for Designing Machine Learning Systemscited by 24
  16. 2020Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projectscited by 24
  17. 2021Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Developmentcited by 21
  18. 2021Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systemscited by 19
  19. 2016On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systemscited by 18
  20. 2021Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approachcited by 18
  21. 2022Symphony: Composing Interactive Interfaces for Machine Learningcited by 18
  22. 2020Towards Guidelines for Assessing Qualities of Machine Learning Systemscited by 16
  23. 2020Software engineering for artificial intelligence and machine learning software: A systematic literature reviewcited by 15
  24. 2019Machine Learning Software Engineering in Practice: An Industrial Case Studycited by 12
  25. 2021Adapting Software Architectures to Machine Learning Challengescited by 11
  26. 2021Practices for Engineering Trustworthy Machine Learning Applicationscited by 8
  27. 2020Towards evaluating and eliciting high-quality documentation for intelligent systemscited by 7
  28. 2021On Misbehaviour and Fault Tolerance in Machine Learning Systemscited by 6
  29. 2019Software Engineering Practice in the Development of Deep Learning Applicationscited by 5
  30. 2019Measuring Non-Expert Comprehension of Machine Learning Fairness Metricscited by 4
  31. 2021Machine Learning Application Development: Practitioners' Insightscited by 2
  32. 2022What is Software Quality for AI Engineers? Towards a Thinning of the Fogcited by 2
  33. 2020A Systematic Literature Review of Modern Software Visualizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 7; 2026: 4.

22 papers.

Sort:

By area:

  1. 2024Prompts Are Programs Too! Understanding How Developers Build Software Containing Promptscited by 19
  2. 2024Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Productscited by 11
  3. 2023Beyond Testers' Biases: Guiding Model Testing with Knowledge Bases using LLMscited by 9
  4. 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systemscited by 6
  5. 2023Test & Evaluation Best Practices for Machine Learning-Enabled Systemscited by 4
  6. 2024How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detectioncited by 3
  7. 2024Naming the Pain in Machine Learning-Enabled Systems Engineeringcited by 3
  8. 2024Innovating for Tomorrow: The Convergence of SE and Green AIcited by 3
  9. 2026A Reference Architecture of Reinforcement Learning Frameworkscited by 3
  10. 2024What About the Data? A Mapping Study on Data Engineering for AI Systemscited by 2
  11. 2025Navigating Pitfalls: Evaluating LLMs in Machine Learning Programming Educationcited by 2
  12. 2026Agentic Pipelines in Embedded Software Engineering: Emerging Practices and Challengescited by 2
10 more
  1. 2026How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Studycited by 2
  2. 2025Agile Management for Machine Learning: A Systematic Mapping Studycited by 1
  3. 2025Understanding Prompt Programming Tasks and Questionscited by 1
  4. 2023Towards a Prediction of Machine Learning Training Time to Support Continuous Learning Systems Developmentcited by 0
  5. 2024Using Quality Attribute Scenarios for ML Model Test Case Generationcited by 0
  6. 2025Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Developmentcited by 0
  7. 2025Define-ML: An Approach to Ideate Machine Learning-Enabled Systemscited by 0
  8. 2025Expanding ML-Documentation Standards For Better Securitycited by 0
  9. 2025Enhancing software product lines with machine learning componentscited by 0
  10. 2026Architecturally Significant MLOps Guidelines for ML Model Integration and Deployment: a Gray Literature Reviewcited by 0