# Assessing the Use of AutoML for Data-Driven Software Engineering

arXiv: https://arxiv.org/abs/2307.10774 · HTML: https://mlsanity.com/paper/2307.10774/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [An Open Source AutoML Benchmark](https://mlsanity.com/paper/1907.00909/) (2019)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [The AI Index 2021 Annual Report](https://mlsanity.com/paper/2103.06312/) (2021)
- [Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows](https://mlsanity.com/paper/2101.04834/) (2021)
- [Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop](https://mlsanity.com/paper/2101.04296/) (2021)
- [Benchmarking Automatic Machine Learning Frameworks](https://mlsanity.com/paper/1808.06492/) (2018)
- [Sentiment Polarity Detection for Software Development](https://mlsanity.com/paper/1709.02984/) (2017)
- [AutoML in The Wild: Obstacles, Workarounds, and Expectations](https://mlsanity.com/paper/2302.10827/) (2023)
- [Can We Use SE-specific Sentiment Analysis Tools in a Cross-Platform Setting?](https://mlsanity.com/paper/2004.00300/) (2020)
- [Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark](https://mlsanity.com/paper/2009.01564/) (2020)
- [An Empirical Study on the Usage of Automated Machine Learning Tools](https://mlsanity.com/paper/2208.13116/) (2022)

## Built on by (most cited first, 2 of 2)

- [DREAM: Debugging and Repairing AutoML Pipelines](https://mlsanity.com/paper/2401.00379/) (2023)
- [Human-Centered AI Product Prototyping with No-Code AutoML: Conceptual Framework, Potentials and Limitations](https://mlsanity.com/paper/2402.07933/) (2024)

## Abstract (arXiv)

Background. Due to the widespread adoption of Artificial Intelligence (AI) and Machine Learning (ML) for building software applications, companies are struggling to recruit employees with a deep understanding of such technologies. In this scenario, AutoML is soaring as a promising solution to fill the AI/ML skills gap since it promises to automate the building of end-to-end AI/ML pipelines that would normally be engineered by specialized team members. Aims. Despite the growing interest and high expectations, there is a dearth of information about the extent to which AutoML is currently adopted by teams developing AI/ML-enabled systems and how it is perceived by practitioners and researchers. Method. To fill these gaps, in this paper, we present a mixed-method study comprising a benchmark of 12 end-to-end AutoML tools on two SE datasets and a user survey with follow-up interviews to further our understanding of AutoML adoption and perception. Results. We found that AutoML solutions can generate models that outperform those trained and optimized by researchers to perform classification tasks in the SE domain. Also, our findings show that the currently available AutoML solutions do not live up to their names as they do not equally support automation across the stages of the ML development workflow and for all the team members. Conclusions. We derive insights to inform the SE research community on how AutoML can facilitate their activities and tool builders on how to design the next generation of AutoML technologies.