# A Scalable AutoML Approach Based on Graph Neural Networks

arXiv: https://arxiv.org/abs/2111.00083 · HTML: https://mlsanity.com/paper/2111.00083/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Learning Deep Generative Models of Graphs](https://mlsanity.com/paper/1803.03324/) (2018)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [An Open Source AutoML Benchmark](https://mlsanity.com/paper/1907.00909/) (2019)
- [An ADMM Based Framework for AutoML Pipeline Configuration](https://mlsanity.com/paper/1905.00424/) (2019)
- [Probabilistic Matrix Factorization for Automated Machine Learning](https://mlsanity.com/paper/1705.05355/) (2017)
- [OBOE: Collaborative Filtering for AutoML Model Selection](https://mlsanity.com/paper/1808.03233/) (2018)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [A Toolkit for Generating Code Knowledge Graphs](https://mlsanity.com/paper/2002.09440/) (2020)
- [AutoML using Metadata Language Embeddings](https://mlsanity.com/paper/1910.03698/) (2019)
- [Automated Machine Learning with Monte-Carlo Tree Search](https://mlsanity.com/paper/1906.00170/) (2019)
- [Recognizing Variables from their Data via Deep Embeddings of Distributions](https://mlsanity.com/paper/1909.04844/) (2019)

## Built on by (most cited first, 7 of 7)

- [SubStrat: A Subset-Based Strategy for Faster AutoML](https://mlsanity.com/paper/2206.03070/) (2022)
- [KGLiDS: A Platform for Semantic Abstraction, Linking, and Automation of Data Science](https://mlsanity.com/paper/2303.02204/) (2023)
- [Federated Data Science to Break Down Silos [Vision]](https://mlsanity.com/paper/2111.13186/) (2021)
- [Federated Learning for Heterogeneous Electronic Health Record Systems with Cost Effective Participant Selection](https://mlsanity.com/paper/2404.13318/) (2024)
- [Serenity: Library Based Python Code Analysis for Code Completion and Automated Machine Learning](https://mlsanity.com/paper/2301.05108/) (2023)
- [A Vision for Semantically Enriched Data Science](https://mlsanity.com/paper/2303.01378/) (2023)
- [VeML: An End-to-End Machine Learning Lifecycle for Large-scale and High-dimensional Data](https://mlsanity.com/paper/2304.13037/) (2023)

## Abstract (arXiv)

AutoML systems build machine learning models automatically by performing a search over valid data transformations and learners, along with hyper-parameter optimization for each learner. Many AutoML systems use meta-learning to guide search for optimal pipelines. In this work, we present a novel meta-learning system called KGpip which, (1) builds a database of datasets and corresponding pipelines by mining thousands of scripts with program analysis, (2) uses dataset embeddings to find similar datasets in the database based on its content instead of metadata-based features, (3) models AutoML pipeline creation as a graph generation problem, to succinctly characterize the diverse pipelines seen for a single dataset. KGpip's meta-learning is a sub-component for AutoML systems. We demonstrate this by integrating KGpip with two AutoML systems. Our comprehensive evaluation using 126 datasets, including those used by the state-of-the-art systems, shows that KGpip significantly outperforms these systems.