# Designing Machine Learning Toolboxes: Concepts, Principles and Patterns

arXiv: https://arxiv.org/abs/2101.04938 · HTML: https://mlsanity.com/paper/2101.04938/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 44 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 9 of 9)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [sktime: A Unified Interface for Machine Learning with Time Series](https://mlsanity.com/paper/1909.07872/) (2019)
- [Software engineering for artificial intelligence and machine learning software: A systematic literature review](https://mlsanity.com/paper/2011.03751/) (2020)
- [MLJ: A Julia package for composable machine learning](https://mlsanity.com/paper/2007.12285/) (2020)
- [Learning Reductions that Really Work](https://mlsanity.com/paper/1502.02704/) (2015)
- [Object-Oriented Programming, Functional Programming and R](https://mlsanity.com/paper/1409.3531/) (2014)
- [Forecasting with sktime: Designing sktime's New Forecasting API and Applying It to Replicate and Extend the M4 Study](https://mlsanity.com/paper/2005.08067/) (2020)
- [mlr3proba: An R Package for Machine Learning in Survival Analysis](https://mlsanity.com/paper/2008.08080/) (2020)

## Built on by (most cited first, 2 of 2)

- [Improving the Learnability of Machine Learning APIs by Semi-Automated API Wrapping](https://mlsanity.com/paper/2203.15491/) (2022)
- [OutlierDetection.jl: A modular outlier detection ecosystem for the Julia programming language](https://mlsanity.com/paper/2211.04550/) (2022)

## Abstract (arXiv)

Machine learning (ML) and AI toolboxes such as scikit-learn or Weka are workhorses of contemporary data scientific practice -- their central role being enabled by usable yet powerful designs that allow to easily specify, train and validate complex modeling pipelines. However, despite their universal success, the key design principles in their construction have never been fully analyzed. In this paper, we attempt to provide an overview of key patterns in the design of AI modeling toolboxes, taking inspiration, in equal parts, from the field of software engineering, implementation patterns found in contemporary toolboxes, and our own experience from developing ML toolboxes. In particular, we develop a conceptual model for the AI/ML domain, with a new type system, called scientific types, at its core. Scientific types capture the scientific meaning of common elements in ML workflows based on the set of operations that we usually perform with them (i.e. their interface) and their statistical properties. From our conceptual analysis, we derive a set of design principles and patterns. We illustrate that our analysis can not only explain the design of existing toolboxes, but also guide the development of new ones. We intend our contribution to be a state-of-art reference for future toolbox engineers, a summary of best practices, a collection of ML design patterns which may become useful for future research, and, potentially, the first steps towards a higher-level programming paradigm for constructing AI.