# SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions

arXiv: https://arxiv.org/abs/2202.10451 · HTML: https://mlsanity.com/paper/2202.10451/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 52 references that are in the map.
- First posted on arXiv 18 Feb 2022.

## Builds on (most cited first, 12 of 12)

- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Feature Engineering for Predictive Modeling using Reinforcement Learning](https://mlsanity.com/paper/1709.07150/) (2017)
- [Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science](https://mlsanity.com/paper/1603.06212/) (2016)
- [Getafix: Learning to Fix Bugs Automatically](https://mlsanity.com/paper/1902.06111/) (2019)
- [OBOE: Collaborative Filtering for AutoML Model Selection](https://mlsanity.com/paper/1808.03233/) (2018)
- [A Grammar-Based Structural CNN Decoder for Code Generation](https://mlsanity.com/paper/1811.06837/) (2018)
- [Aroma: Code Recommendation via Structural Code Search](https://mlsanity.com/paper/1812.01158/) (2018)
- [Program Synthesis using Conflict-Driven Learning](https://mlsanity.com/paper/1711.08029/) (2017)
- [Component-based Synthesis of Table Consolidation and Transformation Tasks from Examples](https://mlsanity.com/paper/1611.07502/) (2016)
- [Neural Programming by Example](https://mlsanity.com/paper/1703.04990/) (2017)
- [ReinBo: Machine Learning pipeline search and configuration with Bayesian Optimization embedded Reinforcement Learning](https://mlsanity.com/paper/1904.05381/) (2019)

## Built on by (most cited first, 3 of 3)

- [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024)
- [Large Language Models Synergize with Automated Machine Learning](https://mlsanity.com/paper/2405.03727/) (2024)
- [ATLO-ML: Adaptive Time-Length Optimizer for Machine Learning -- Insights from Air Quality Forecasting](https://mlsanity.com/paper/2510.06503/) (2025)

## Abstract (arXiv)

Automatic machine learning, or AutoML, holds the promise of truly democratizing the use of machine learning (ML), by substantially automating the work of data scientists. However, the huge combinatorial search space of candidate pipelines means that current AutoML techniques, generate sub-optimal pipelines, or none at all, especially on large, complex datasets. In this work we propose an AutoML technique SapientML, that can learn from a corpus of existing datasets and their human-written pipelines, and efficiently generate a high-quality pipeline for a predictive task on a new dataset. To combat the search space explosion of AutoML, SapientML employs a novel divide-and-conquer strategy realized as a three-stage program synthesis approach, that reasons on successively smaller search spaces. The first stage uses a machine-learned model to predict a set of plausible ML components to constitute a pipeline. In the second stage, this is then refined into a small pool of viable concrete pipelines using syntactic constraints derived from the corpus and the machine-learned model. Dynamically evaluating these few pipelines, in the third stage, provides the best solution. We instantiate SapientML as part of a fully automated tool-chain that creates a cleaned, labeled learning corpus by mining Kaggle, learns from it, and uses the learned models to then synthesize pipelines for new predictive tasks. We have created a training corpus of 1094 pipelines spanning 170 datasets, and evaluated SapientML on a set of 41 benchmark datasets, including 10 new, large, real-world datasets from Kaggle, and against 3 state-of-the-art AutoML tools and 2 baselines. Our evaluation shows that SapientML produces the best or comparable accuracy on 27 of the benchmarks while the second best tool fails to even produce a pipeline on 9 of the instances.