# The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development

arXiv: https://arxiv.org/abs/1905.08942 · HTML: https://mlsanity.com/paper/1905.08942/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 13 of 13)

- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [Sintel: A Machine Learning Framework to Extract Insights from Signals](https://mlsanity.com/paper/2204.09108/) (2022)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [Targeted synthetic data generation for tabular data via hardness characterization](https://mlsanity.com/paper/2410.00759/) (2024)
- [MEDS-Tab: Automated tabularization and baseline methods for MEDS datasets](https://mlsanity.com/paper/2411.00200/) (2024)
- [AutoMLBench: A Comprehensive Experimental Evaluation of Automated Machine Learning Frameworks](https://mlsanity.com/paper/2204.08358/) (2022)
- [TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data](https://mlsanity.com/paper/2111.04798/) (2021)
- [OrionBench: Benchmarking Time Series Generative Models in the Service of the End-User](https://mlsanity.com/paper/2310.17748/) (2023)
- [A Grammar of Machine Learning Workflows: Rejecting Data Leakage at Call Time](https://mlsanity.com/paper/2603.10742/) (2026)
- [Efficient End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2206.09423/) (2022)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)

## Abstract (arXiv)

As machine learning is applied more widely, data scientists often struggle to find or create end-to-end machine learning systems for specific tasks. The proliferation of libraries and frameworks and the complexity of the tasks have led to the emergence of "pipeline jungles" - brittle, ad hoc ML systems. To address these problems, we introduce the Machine Learning Bazaar, a new framework for developing machine learning and automated machine learning software systems. First, we introduce ML primitives, a unified API and specification for data processing and ML components from different software libraries. Next, we compose primitives into usable ML pipelines, abstracting away glue code, data flow, and data storage. We further pair these pipelines with a hierarchy of AutoML strategies - Bayesian optimization and bandit learning. We use these components to create a general-purpose, multi-task, end-to-end AutoML system that provides solutions to a variety of data modalities (image, text, graph, tabular, relational, etc.) and problem types (classification, regression, anomaly detection, graph matching, etc.). We demonstrate 5 real-world use cases and 2 case studies of our approach. Finally, we present an evaluation suite of 456 real-world ML tasks and describe the characteristics of 2.5 million pipelines searched over this task suite.