# PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines

arXiv: https://arxiv.org/abs/2005.00160 · HTML: https://mlsanity.com/paper/2005.00160/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 19 of 19)

- [Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows](https://mlsanity.com/paper/2101.04834/) (2021)
- [Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop](https://mlsanity.com/paper/2101.04296/) (2021)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning](https://mlsanity.com/paper/2202.11954/) (2022)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments](https://mlsanity.com/paper/2408.03274/) (2024)
- [Data+Shift: Supporting visual investigation of data distribution shifts by data scientists](https://mlsanity.com/paper/2204.14025/) (2022)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines](https://mlsanity.com/paper/2504.13587/) (2025)
- [DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning](https://mlsanity.com/paper/2512.01810/) (2025)
- [AgentTrails: Towards Trust and Reuse for Agentic Tasks](https://mlsanity.com/paper/2607.18816/) (2026)
- [Integration Of Evolutionary Automated Machine Learning With Structural Sensitivity Analysis For Composite Pipelines](https://mlsanity.com/paper/2312.14770/) (2023)
- [Evolving machine learning workflows through interactive AutoML](https://mlsanity.com/paper/2402.18505/) (2024)
- [A Comparative Visual Analytics Framework for Evaluating Evolutionary Processes in Multi-objective Optimization](https://mlsanity.com/paper/2308.05640/) (2023)
- [Deciphering AutoML Ensembles: cattleia's Assistance in Decision-Making](https://mlsanity.com/paper/2403.12664/) (2024)

## Abstract (arXiv)

In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to search and generate end-to-end learning pipelines. While these techniques facilitate the creation of models for real-world applications, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipelines they derive, it is difficult for their developers to debug these systems. It is also challenging for machine learning experts to select an AutoML system that is well suited for a given problem or class of problems. In this paper, we present the PipelineProfiler, an interactive visualization tool that allows the exploration and comparison of the solution space of machine learning (ML) pipelines produced by AutoML systems. PipelineProfiler is integrated with Jupyter Notebook and can be used together with common data science tools to enable a rich set of analyses of the ML pipelines and provide insights about the algorithms that generated them. We demonstrate the utility of our tool through several use cases where PipelineProfiler is used to better understand and improve a real-world AutoML system. Furthermore, we validate our approach by presenting a detailed analysis of a think-aloud experiment with six data scientists who develop and evaluate AutoML tools.