# Efficient End-to-End AutoML via Scalable Search Space Decomposition

arXiv: https://arxiv.org/abs/2206.09423 · HTML: https://mlsanity.com/paper/2206.09423/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 108 references that are in the map.
- First posted on arXiv 19 Jun 2022.

## Builds on (most cited first, 30 of 42)

- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [Feature Engineering for Predictive Modeling using Reinforcement Learning](https://mlsanity.com/paper/1709.07150/) (2017)
- [Hyperparameter Importance Across Datasets](https://mlsanity.com/paper/1710.04725/) (2017)
- [Automated Machine Learning: From Principles to Practices](https://mlsanity.com/paper/1810.13306/) (2018)
- [Rotting Bandits](https://mlsanity.com/paper/1702.07274/) (2017)
- [An ADMM Based Framework for AutoML Pipeline Configuration](https://mlsanity.com/paper/1905.00424/) (2019)
- [OpenBox: A Generalized Black-box Optimization Service](https://mlsanity.com/paper/2106.00421/) (2021)
- [Efficient Automatic CASH via Rising Bandits](https://mlsanity.com/paper/2012.04371/) (2020)
- [ZeroER: Entity Resolution using Zero Labeled Examples](https://mlsanity.com/paper/1908.06049/) (2019)
- [Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit](https://mlsanity.com/paper/1702.00403/) (2017)
- [AlphaD3M: Machine Learning Pipeline Synthesis](https://mlsanity.com/paper/2111.02508/) (2021)
- [Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation](https://mlsanity.com/paper/2110.12654/) (2021)
- [Multi-fidelity Bayesian Optimization with Max-value Entropy Search and its parallelization](https://mlsanity.com/paper/1901.08275/) (2019)
- [SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle](https://mlsanity.com/paper/1909.02976/) (2019)
- [Autostacker: A Compositional Evolutionary Learning System](https://mlsanity.com/paper/1803.00684/) (2018)
- [Benchmark and Survey of Automated Machine Learning Frameworks](https://mlsanity.com/paper/1904.12054/) (2019)

## Abstract (arXiv)

End-to-end AutoML has attracted intensive interests from both academia and industry which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning. Existing AutoML systems, however, suffer from scalability issues when applying to application domains with large, high-dimensional search spaces. We present VolcanoML, a scalable and extensible framework that facilitates systematic exploration of large AutoML search spaces. VolcanoML introduces and implements basic building blocks that decompose a large search space into smaller ones, and allows users to utilize these building blocks to compose an execution plan for the AutoML problem at hand. VolcanoML further supports a Volcano-style execution model -- akin to the one supported by modern database systems -- to execute the plan constructed. Our evaluation demonstrates that, not only does VolcanoML raise the level of expressiveness for search space decomposition in AutoML, it also leads to actual findings of decomposition strategies that are significantly more efficient than the ones employed by state-of-the-art AutoML systems such as auto-sklearn. This paper is the extended version of the initial VolcanoML paper appeared in VLDB 2021.