# Modeling Scalability of Distributed Machine Learning

arXiv: https://arxiv.org/abs/1610.06276 · HTML: https://mlsanity.com/paper/1610.06276/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2016.

## Built on by (most cited first, 1 of 1)

- [A Conceptual Framework for AI-based Decision Systems in Critical Infrastructures](https://mlsanity.com/paper/2504.16133/) (2025)

## Abstract (arXiv)

Present day machine learning is computationally intensive and processes large amounts of data. It is implemented in a distributed fashion in order to address these scalability issues. The work is parallelized across a number of computing nodes. It is usually hard to estimate in advance how many nodes to use for a particular workload. We propose a simple framework for estimating the scalability of distributed machine learning algorithms. We measure the scalability by means of the speedup an algorithm achieves with more nodes. We propose time complexity models for gradient descent and graphical model inference. We validate our models with experiments on deep learning training and belief propagation. This framework was used to study the scalability of machine learning algorithms in Apache Spark.