# Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows

arXiv: https://arxiv.org/abs/2101.04834 · HTML: https://mlsanity.com/paper/2101.04834/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Auto-Keras: An Efficient Neural Architecture Search System](https://mlsanity.com/paper/1806.10282/) (2018)
- [A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective](https://mlsanity.com/paper/1811.03402/) (2018)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [AdaNet: Adaptive Structural Learning of Artificial Neural Networks](https://mlsanity.com/paper/1607.01097/) (2016)
- [Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems](https://mlsanity.com/paper/2001.06509/) (2020)
- [Ludwig: a type-based declarative deep learning toolbox](https://mlsanity.com/paper/1909.07930/) (2019)
- [AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates](https://mlsanity.com/paper/1912.06723/) (2019)
- [ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning](https://mlsanity.com/paper/1902.05009/) (2019)
- [PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines](https://mlsanity.com/paper/2005.00160/) (2020)
- [Serverless Computing: Current Trends and Open Problems](https://mlsanity.com/paper/1706.03178/) (2017)
- [Helix: Holistic Optimization for Accelerating Iterative Machine Learning](https://mlsanity.com/paper/1812.05762/) (2018)
- [Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development](https://mlsanity.com/paper/2005.01520/) (2020)

## Built on by (most cited first, 22 of 22)

- [Operationalizing Machine Learning: An Interview Study](https://mlsanity.com/paper/2209.09125/) (2022)
- [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024)
- [IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective](https://mlsanity.com/paper/2209.08018/) (2022)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work](https://mlsanity.com/paper/2203.11085/) (2022)
- [Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"](https://mlsanity.com/paper/2302.01448/) (2023)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines](https://mlsanity.com/paper/2106.15397/) (2021)
- [Large Language Models Synergize with Automated Machine Learning](https://mlsanity.com/paper/2405.03727/) (2024)
- [AutoML in The Wild: Obstacles, Workarounds, and Expectations](https://mlsanity.com/paper/2302.10827/) (2023)
- [Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling](https://mlsanity.com/paper/2304.08366/) (2023)
- [An Empirical Study on the Usage of Automated Machine Learning Tools](https://mlsanity.com/paper/2208.13116/) (2022)
- [HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization](https://mlsanity.com/paper/2502.01276/) (2025)
- [RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines](https://mlsanity.com/paper/2504.13587/) (2025)
- [Human-Centered AI for Data Science: A Systematic Approach](https://mlsanity.com/paper/2110.01108/) (2021)
- [Assessing the Use of AutoML for Data-Driven Software Engineering](https://mlsanity.com/paper/2307.10774/) (2023)
- [LoCoML: A Framework for Real-World ML Inference Pipelines](https://mlsanity.com/paper/2501.14165/) (2025)
- [Practitioner Motives to Use Different Hyperparameter Optimization Methods](https://mlsanity.com/paper/2203.01717/) (2022)
- [EDCA - An Evolutionary Data-Centric AutoML Framework for Efficient Pipelines](https://mlsanity.com/paper/2503.04350/) (2025)
- [Predicting delays in Indian lower courts using AutoML and Decision Forests](https://mlsanity.com/paper/2307.16285/) (2023)
- [Eliciting Model Steering Interactions from Users via Data and Visual Design Probes](https://mlsanity.com/paper/2310.09314/) (2023)
- [Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems](https://mlsanity.com/paper/2609.38552/) (2026)

## Abstract (arXiv)

Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.