Position: A Call to Action for a Human-Centered AutoML Paradigm
Not on the map. Why?
Abstract (arXiv)
Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience. Over the past decade, commendable achievements in AutoML have primarily focused on optimizing predictive performance. This focused progress, while substantial, raises questions about how well AutoML has met its broader, original goals. In this position paper, we argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems, including their diverse roles, expectations, and expertise. We envision a more human-centered approach in future AutoML research, promoting the collaborative design of ML systems that tightly integrates the complementary strengths of human expertise and AutoML methodologies.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 70 of its 131 references that are in the map. First posted on arXiv 5 Jun 2024.
Builds on
70 of 131 references.
References · 70
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
- 2016The Mythos of Model Interpretabilitycited by 716
- 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
- 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
- 2016Learning to learn by gradient descent by gradient descentcited by 616
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
- 2023Symbolic Discovery of Optimization Algorithmscited by 423
- 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 368
- 2019The many Shapley values for model explanationcited by 341
38 more
- 2020A Survey on Neural Network Interpretabilitycited by 291
- 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
- 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
- 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
- 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
- 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 212
- 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
- 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
- 2020Multi-Objective Counterfactual Explanationscited by 157
- 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 137
- 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
- 2019FLAML: A Fast and Lightweight AutoML Librarycited by 104
- 2018Tunability: Importance of Hyperparameters of Machine Learning Algorithmscited by 98
- 2019Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importancecited by 85
- 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 84
- 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 80
- 2019Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AIcited by 79
- 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 70
- 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 69
- 2020AutoML to Date and Beyond: Challenges and Opportunitiescited by 63
- 2017Hyperparameter Importance Across Datasetscited by 62
- 2018Visualizing the Feature Importance for Black Box Modelscited by 55
- 2018Bayesian Optimization in AlphaGocited by 55
- 2020Fair Bayesian Optimizationcited by 45
- 2020Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodologycited by 41
- 2022Automated Dynamic Algorithm Configurationcited by 37
- 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
- 2020Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systemscited by 36
- 2021Explaining Hyperparameter Optimization via Partial Dependence Plotscited by 35
- 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
- 2023PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learningcited by 31
- 2018AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learningcited by 22
- 2019Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretabilitycited by 22
- 2021Whither AutoML? Understanding the Role of Automation in Machine Learning Workflowscited by 22
- 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
- 2021Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loopcited by 21
- 2021How Much Automation Does a Data Scientist Want?cited by 20
- 2020PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelinescited by 19
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2024: 3; 2025: 8; 2026: 3.
14 papers.
Sort:
By area:
- 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
- 2025A Statistical Case Against Empirical Human-AI Alignmentcited by 6
- 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
- 2025HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimizationcited by 4
- 2025Hyperparameter Optimization via Interacting with Probabilistic Circuitscited by 4
- 2024CliMB: An AI-enabled Partner for Clinical Predictive Modelingcited by 2
- 2025Dynamic Priors in Bayesian Optimization for Hyperparameter Optimizationcited by 2
- 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learningcited by 2
- 2025Dataset-Agnostic Recommender Systemscited by 1
- 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0
- 2025Optimization Problem Solving Can Transition to Evolutionary Agentic Workflowscited by 0
- 2026Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clusteringcited by 0
- 2026DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance29 Sep 2026
- 2026SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensembling11 Sep 2026
- 2026Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clustering20 Feb 2026
- 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning1 Dec 2025
- 2025Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization4 Nov 2025
- 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings8 Jun 2025
- 2025Hyperparameter Optimization via Interacting with Probabilistic Circuits23 May 2025
- 2025Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows7 May 2025
- 2025A Statistical Case Against Empirical Human-AI Alignment20 Feb 2025
- 2025HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization3 Feb 2025
- 2025Dataset-Agnostic Recommender Systems13 Jan 2025
- 2024A Research Agenda for Usability and Generalisation in Reinforcement Learning22 Dec 2024
- 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
- 2025A Statistical Case Against Empirical Human-AI Alignmentcited by 6
- 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
- 2025HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimizationcited by 4
- 2025Hyperparameter Optimization via Interacting with Probabilistic Circuitscited by 4
- 2024CliMB: An AI-enabled Partner for Clinical Predictive Modelingcited by 2
- 2025Dynamic Priors in Bayesian Optimization for Hyperparameter Optimizationcited by 2
- 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learningcited by 2
- 2025Dataset-Agnostic Recommender Systemscited by 1
- 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0
- 2026Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clusteringcited by 0
- 2026SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensemblingcited by 0
12 most cited of 13 All 13 (JSON)
- 2026DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance29 Sep 2026
- 2026SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensembling11 Sep 2026
- 2026Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clustering20 Feb 2026
- 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning1 Dec 2025
- 2025Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization4 Nov 2025
- 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings8 Jun 2025
- 2025Hyperparameter Optimization via Interacting with Probabilistic Circuits23 May 2025
- 2025A Statistical Case Against Empirical Human-AI Alignment20 Feb 2025
- 2025HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization3 Feb 2025
- 2025Dataset-Agnostic Recommender Systems13 Jan 2025
- 2024A Research Agenda for Usability and Generalisation in Reinforcement Learning22 Dec 2024
- 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey11 Nov 2024
12 newest of 13 All 13 (JSON)