# Reasonable Scale Machine Learning with Open-Source Metaflow

arXiv: https://arxiv.org/abs/2303.11761 · HTML: https://mlsanity.com/paper/2303.11761/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 68 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [Semantic Product Search](https://mlsanity.com/paper/1907.00937/) (2019)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models](https://mlsanity.com/paper/2209.05310/) (2022)
- [Operationalizing Machine Learning: An Interview Study](https://mlsanity.com/paper/2209.09125/) (2022)
- [Ludwig: a type-based declarative deep learning toolbox](https://mlsanity.com/paper/1909.07930/) (2019)
- [SIGIR 2021 E-Commerce Workshop Data Challenge](https://mlsanity.com/paper/2104.09423/) (2021)
- [Query2Prod2Vec Grounded Word Embeddings for eCommerce](https://mlsanity.com/paper/2104.02061/) (2021)
- [Beyond NDCG: behavioral testing of recommender systems with RecList](https://mlsanity.com/paper/2111.09963/) (2021)
- [BERT Goes Shopping: Comparing Distributional Models for Product Representations](https://mlsanity.com/paper/2012.09807/) (2020)
- [You Do Not Need a Bigger Boat: Recommendations at Reasonable Scale in a (Mostly) Serverless and Open Stack](https://mlsanity.com/paper/2107.07346/) (2021)
- [DAG Card is the new Model Card](https://mlsanity.com/paper/2110.13601/) (2021)
- ["Are you sure?": Preliminary Insights from Scaling Product Comparisons to Multiple Shops](https://mlsanity.com/paper/2107.03256/) (2021)
- ["Does it come in black?" CLIP-like models are zero-shot recommenders](https://mlsanity.com/paper/2204.02473/) (2022)
- [EvalRS: a Rounded Evaluation of Recommender Systems](https://mlsanity.com/paper/2207.05772/) (2022)
- [Infrastructure for Usable Machine Learning: The Stanford DAWN Project](https://mlsanity.com/paper/1705.07538/) (2017)

## Built on by (most cited first, 5 of 5)

- [Safe, Untrusted, "Proof-Carrying" AI Agents: toward the agentic lakehouse](https://mlsanity.com/paper/2510.09567/) (2025)
- [Reproducible data science over data lakes: replayable data pipelines with Bauplan and Nessie](https://mlsanity.com/paper/2404.13682/) (2024)
- [JetTrain: IDE-Native Machine Learning Experiments](https://mlsanity.com/paper/2402.10857/) (2024)
- [(Vector) Space is Not the Final Frontier: Product Search as Program Synthesis](https://mlsanity.com/paper/2304.11473/) (2023)
- [A Configuration-First Framework for Reproducible, Low-Code Machine Learning: a Localization Use Case](https://mlsanity.com/paper/2510.25692/) (2025)

## Abstract (arXiv)

As Machine Learning (ML) gains adoption across industries and new use cases, practitioners increasingly realize the challenges around effectively developing and iterating on ML systems: reproducibility, debugging, scalability, and documentation are elusive goals for real-world pipelines outside tech-first companies. In this paper, we review the nature of ML-oriented workloads and argue that re-purposing existing tools won't solve the current productivity issues, as ML peculiarities warrant specialized development tooling. We then introduce Metaflow, an open-source framework for ML projects explicitly designed to boost the productivity of data practitioners by abstracting away the execution of ML code from the definition of the business logic. We show how our design addresses the main challenges in ML operations (MLOps), and document through examples, interviews and use cases its practical impact on the field.