# Benchmark and Survey of Automated Machine Learning Frameworks

arXiv: https://arxiv.org/abs/1904.12054 · HTML: https://mlsanity.com/paper/1904.12054/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 18 of 18)

- [AutoDS: Towards Human-Centered Automation of Data Science](https://mlsanity.com/paper/2101.05273/) (2021)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [How Much Automation Does a Data Scientist Want?](https://mlsanity.com/paper/2101.03970/) (2021)
- [Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML](https://mlsanity.com/paper/2104.04375/) (2021)
- [Learning Deep Morphological Networks with Neural Architecture Search](https://mlsanity.com/paper/2106.07714/) (2021)
- [AutoML for Large Capacity Modeling of Meta's Ranking Systems](https://mlsanity.com/paper/2311.07870/) (2023)
- [Self-directed Machine Learning](https://mlsanity.com/paper/2201.01289/) (2022)
- [Exploring Opportunistic Meta-knowledge to Reduce Search Spaces for Automated Machine Learning](https://mlsanity.com/paper/2105.00282/) (2021)
- [CUAB: Convolutional Uncertainty Attention Block Enhanced the Chest X-ray Image Analysis](https://mlsanity.com/paper/2105.01840/) (2021)
- [Automated Machine Learning Techniques for Data Streams](https://mlsanity.com/paper/2106.07317/) (2021)
- [Bayesian Optimization and Deep Learning forsteering wheel angle prediction](https://mlsanity.com/paper/2110.13629/) (2021)
- [Sampling Streaming Data with Parallel Vector Quantization -- PVQ](https://mlsanity.com/paper/2210.01792/) (2022)
- [Interpret-able feedback for AutoML systems](https://mlsanity.com/paper/2102.11267/) (2021)
- [Designing Machine Learning Pipeline Toolkit for AutoML Surrogate Modeling Optimization](https://mlsanity.com/paper/2107.01253/) (2021)
- [Efficient End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2206.09423/) (2022)
- [On Taking Advantage of Opportunistic Meta-knowledge to Reduce Configuration Spaces for Automated Machine Learning](https://mlsanity.com/paper/2208.04376/) (2022)
- [AutoEn: An AutoML method based on ensembles of predefined Machine Learning pipelines for supervised Traffic Forecasting](https://mlsanity.com/paper/2303.10732/) (2023)

## Abstract (arXiv)

Machine learning (ML) has become a vital part in many aspects of our daily life. However, building well performing machine learning applications requires highly specialized data scientists and domain experts. Automated machine learning (AutoML) aims to reduce the demand for data scientists by enabling domain experts to build machine learning applications automatically without extensive knowledge of statistics and machine learning. This paper is a combination of a survey on current AutoML methods and a benchmark of popular AutoML frameworks on real data sets. Driven by the selected frameworks for evaluation, we summarize and review important AutoML techniques and methods concerning every step in building an ML pipeline. The selected AutoML frameworks are evaluated on 137 data sets from established AutoML benchmark suits.