# Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems

arXiv: https://arxiv.org/abs/2001.06509 · HTML: https://mlsanity.com/paper/2001.06509/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2020.

## Built on by (most cited first, 30 of 36)

- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Explaining Hyperparameter Optimization via Partial Dependence Plots](https://mlsanity.com/paper/2111.04820/) (2021)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows](https://mlsanity.com/paper/2101.04834/) (2021)
- [Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop](https://mlsanity.com/paper/2101.04296/) (2021)
- [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024)
- [D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias](https://mlsanity.com/paper/2208.05126/) (2022)
- [Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work](https://mlsanity.com/paper/2203.11085/) (2022)
- [DeepCAVE: An Interactive Analysis Tool for Automated Machine Learning](https://mlsanity.com/paper/2206.03493/) (2022)
- [Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application](https://mlsanity.com/paper/2305.08598/) (2023)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture](https://mlsanity.com/paper/2305.12710/) (2023)
- [Naive Automated Machine Learning](https://mlsanity.com/paper/2111.14514/) (2021)
- [XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning](https://mlsanity.com/paper/2202.11954/) (2022)
- [AutoML in The Wild: Obstacles, Workarounds, and Expectations](https://mlsanity.com/paper/2302.10827/) (2023)
- [Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling](https://mlsanity.com/paper/2304.08366/) (2023)
- [From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems](https://mlsanity.com/paper/2502.07974/) (2025)
- [How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions](https://mlsanity.com/paper/2409.07192/) (2024)
- [An Empirical Study on the Usage of Automated Machine Learning Tools](https://mlsanity.com/paper/2208.13116/) (2022)
- [On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Study](https://mlsanity.com/paper/2311.14014/) (2023)
- [HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization](https://mlsanity.com/paper/2502.01276/) (2025)
- [Human-Centered AI for Data Science: A Systematic Approach](https://mlsanity.com/paper/2110.01108/) (2021)
- [DCNFIS: Deep Convolutional Neuro-Fuzzy Inference System](https://mlsanity.com/paper/2308.06378/) (2023)
- [Towards certifiable AI in aviation: landscape, challenges, and opportunities](https://mlsanity.com/paper/2409.08666/) (2024)
- [DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning](https://mlsanity.com/paper/2512.01810/) (2025)
- [Naive Automated Machine Learning -- A Late Baseline for AutoML](https://mlsanity.com/paper/2103.10496/) (2021)
- [Automatic Componentwise Boosting: An Interpretable AutoML System](https://mlsanity.com/paper/2109.05583/) (2021)
- [Practitioner Motives to Use Different Hyperparameter Optimization Methods](https://mlsanity.com/paper/2203.01717/) (2022)
- [Integration Of Evolutionary Automated Machine Learning With Structural Sensitivity Analysis For Composite Pipelines](https://mlsanity.com/paper/2312.14770/) (2023)

## Abstract (arXiv)

We explore trust in a relatively new area of data science: Automated Machine Learning (AutoML). In AutoML, AI methods are used to generate and optimize machine learning models by automatically engineering features, selecting models, and optimizing hyperparameters. In this paper, we seek to understand what kinds of information influence data scientists' trust in the models produced by AutoML? We operationalize trust as a willingness to deploy a model produced using automated methods. We report results from three studies -- qualitative interviews, a controlled experiment, and a card-sorting task -- to understand the information needs of data scientists for establishing trust in AutoML systems. We find that including transparency features in an AutoML tool increased user trust and understandability in the tool; and out of all proposed features, model performance metrics and visualizations are the most important information to data scientists when establishing their trust with an AutoML tool.