# Adoption and Effects of Software Engineering Best Practices in Machine Learning

arXiv: https://arxiv.org/abs/2007.14130 · HTML: https://mlsanity.com/paper/2007.14130/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 30 of 33)

- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2103.09783/) (2021)
- [Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems](https://mlsanity.com/paper/2103.14101/) (2021)
- [Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System](https://mlsanity.com/paper/2204.07874/) (2022)
- [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023)
- [Requirements Engineering for Machine Learning: A Review and Reflection](https://mlsanity.com/paper/2210.00859/) (2022)
- [Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2411.19463/) (2024)
- [An Experience Report on Machine Learning Reproducibility: Guidance for Practitioners and TensorFlow Model Garden Contributors](https://mlsanity.com/paper/2107.00821/) (2021)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Designing Language Technologies for Social Good: The Road not Taken](https://mlsanity.com/paper/2110.07444/) (2021)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review](https://mlsanity.com/paper/2102.07574/) (2021)
- [A Catalog of Fairness-Aware Practices in Machine Learning Engineering](https://mlsanity.com/paper/2408.16683/) (2024)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [What are the Machine Learning best practices reported by practitioners on Stack Exchange?](https://mlsanity.com/paper/2301.10516/) (2023)
- [MLXP: A Framework for Conducting Replicable Experiments in Python](https://mlsanity.com/paper/2402.13831/) (2024)
- ["Project smells" -- Experiences in Analysing the Software Quality of ML Projects with mllint](https://mlsanity.com/paper/2201.08246/) (2022)
- [What is Software Quality for AI Engineers? Towards a Thinning of the Fog](https://mlsanity.com/paper/2203.12697/) (2022)
- [What About the Data? A Mapping Study on Data Engineering for AI Systems](https://mlsanity.com/paper/2402.05156/) (2024)
- [Assessing the Quality of Computational Notebooks for a Frictionless Transition from Exploration to Production](https://mlsanity.com/paper/2205.11941/) (2022)
- [A Framework to Model ML Engineering Processes](https://mlsanity.com/paper/2404.18531/) (2024)
- [The Rise and Fall(?) of Software Engineering](https://mlsanity.com/paper/2406.10141/) (2024)
- [A Case Study on AI Engineering Practices: Developing an Autonomous Stock Trading System](https://mlsanity.com/paper/2303.13216/) (2023)
- [Towards machine learning guided by best practices](https://mlsanity.com/paper/2305.00233/) (2023)
- [Best Practices for Machine Learning Systems: An Industrial Framework for Analysis and Optimization](https://mlsanity.com/paper/2306.13662/) (2023)
- [Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education](https://mlsanity.com/paper/2411.19304/) (2024)
- [Maturity Framework for Enhancing Machine Learning Quality](https://mlsanity.com/paper/2502.15758/) (2025)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)

## Abstract (arXiv)

The increasing reliance on applications with machine learning (ML) components calls for mature engineering techniques that ensure these are built in a robust and future-proof manner. We aim to empirically determine the state of the art in how teams develop, deploy and maintain software with ML components. We mined both academic and grey literature and identified 29 engineering best practices for ML applications. We conducted a survey among 313 practitioners to determine the degree of adoption for these practices and to validate their perceived effects. Using the survey responses, we quantified practice adoption, differentiated along demographic characteristics, such as geography or team size. We also tested correlations and investigated linear and non-linear relationships between practices and their perceived effect using various statistical models. Our findings indicate, for example, that larger teams tend to adopt more practices, and that traditional software engineering practices tend to have lower adoption than ML specific practices. Also, the statistical models can accurately predict perceived effects such as agility, software quality and traceability, from the degree of adoption for specific sets of practices. Combining practice adoption rates with practice importance, as revealed by statistical models, we identify practices that are important but have low adoption, as well as practices that are widely adopted but are less important for the effects we studied. Overall, our survey and the analysis of responses received provide a quantitative basis for assessment and step-wise improvement of practice adoption by ML teams.