# Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education

arXiv: https://arxiv.org/abs/2411.19304 · HTML: https://mlsanity.com/paper/2411.19304/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 81 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Machine Learning Operations (MLOps): Overview, Definition, and Architecture](https://mlsanity.com/paper/2205.02302/) (2022)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [NatGen: Generative pre-training by "Naturalizing" source code](https://mlsanity.com/paper/2206.07585/) (2022)
- [Software Engineering Challenges of Deep Learning](https://mlsanity.com/paper/1810.12034/) (2018)
- [A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research](https://mlsanity.com/paper/2009.06520/) (2020)
- [How to avoid machine learning pitfalls: a guide for academic researchers](https://mlsanity.com/paper/2108.02497/) (2021)
- [Adoption and Effects of Software Engineering Best Practices in Machine Learning](https://mlsanity.com/paper/2007.14130/) (2020)
- [Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems](https://mlsanity.com/paper/2103.14101/) (2021)
- [Pitfalls in Machine Learning Research: Reexamining the Development Cycle](https://mlsanity.com/paper/2011.02832/) (2020)
- [What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow](https://mlsanity.com/paper/1906.11940/) (2019)
- [Documentation of Machine Learning Software](https://mlsanity.com/paper/2001.11956/) (2020)
- [Practices for Engineering Trustworthy Machine Learning Applications](https://mlsanity.com/paper/2103.00964/) (2021)
- [Teaching Software Engineering for AI-Enabled Systems](https://mlsanity.com/paper/2001.06691/) (2020)
- [Machine Learning for Software Engineering: A Tertiary Study](https://mlsanity.com/paper/2211.09425/) (2022)
- [Towards Integrating Emerging AI Applications in SE Education](https://mlsanity.com/paper/2405.18062/) (2024)
- [Lessons Learned from Educating AI Engineers](https://mlsanity.com/paper/2103.10703/) (2021)
- [What are the Machine Learning best practices reported by practitioners on Stack Exchange?](https://mlsanity.com/paper/2301.10516/) (2023)
- [Teaching Machine Learning in K-12 Computing Education: Potential and Pitfalls](https://mlsanity.com/paper/2106.11034/) (2021)
- [Empirical Standards for Repository Mining](https://mlsanity.com/paper/2203.15950/) (2022)
- [Integrating machine learning concepts into undergraduate classes](https://mlsanity.com/paper/2211.06491/) (2022)

## Abstract (arXiv)

Context: Machine Learning (ML) significantly impacts Software Engineering (SE), but studies mainly focus on practitioners, neglecting researchers. This overlooks practices and challenges in teaching, researching, or reviewing ML applications in SE. Objective: This study aims to contribute to the knowledge, about the synergy between ML and SE from the perspective of SE researchers, by providing insights into the practices followed when researching, teaching, and reviewing SE studies that apply ML. Method: We analyzed SE researchers familiar with ML or who authored SE articles using ML, along with the articles themselves. We examined practices, SE tasks addressed with ML, challenges faced, and reviewers' and educators' perspectives using grounded theory coding and qualitative analysis. Results: We found diverse practices focusing on data collection, model training, and evaluation. Some recommended practices (e.g., hyperparameter tuning) appeared in less than 20% of literature. Common challenges involve data handling, model evaluation (incl. non-functional properties), and involving human expertise in evaluation. Hands-on activities are common in education, though traditional methods persist. Conclusion: Despite accepted practices in applying ML to SE, significant gaps remain. By enhancing guidelines, adopting diverse teaching methods, and emphasizing underrepresented practices, the SE community can bridge these gaps and advance the field.