# Scavenger: A Cloud Service for Optimizing Cost and Performance of ML Training

arXiv: https://arxiv.org/abs/2303.06659 · HTML: https://mlsanity.com/paper/2303.06659/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 12 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/1802.09941/) (2018)
- [Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning](https://mlsanity.com/paper/2008.12260/) (2020)
- [Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools](https://mlsanity.com/paper/1903.11314/) (2019)
- [Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study](https://mlsanity.com/paper/1509.04210/) (2015)
- [Large batch size training of neural networks with adversarial training and second-order information](https://mlsanity.com/paper/1810.01021/) (2018)
- [Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads](https://mlsanity.com/paper/1708.07308/) (2017)
- [AdaScale SGD: A User-Friendly Algorithm for Distributed Training](https://mlsanity.com/paper/2007.05105/) (2020)
- [On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization](https://mlsanity.com/paper/1905.04346/) (2019)
- [HyperSched: Dynamic Resource Reallocation for Model Development on a Deadline](https://mlsanity.com/paper/2001.02338/) (2020)
- [TensorFlow Estimators: Managing Simplicity vs. Flexibility in High-Level Machine Learning Frameworks](https://mlsanity.com/paper/1708.02637/) (2017)
- [BFTrainer: Low-Cost Training of Neural Networks on Unfillable Supercomputer Nodes](https://mlsanity.com/paper/2106.12091/) (2021)
- [EasyScale: Accuracy-consistent Elastic Training for Deep Learning](https://mlsanity.com/paper/2208.14228/) (2022)

## Built on by (most cited first, 5 of 5)

- [GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training](https://mlsanity.com/paper/2305.12201/) (2023)
- [OmniLearn: A Framework for Distributed Deep Learning over Heterogeneous Clusters](https://mlsanity.com/paper/2503.17469/) (2025)
- [Flexible Communication for Optimal Distributed Learning over Unpredictable Networks](https://mlsanity.com/paper/2312.02493/) (2023)
- [On Using Large-Batches in Federated Learning](https://mlsanity.com/paper/2509.10537/) (2025)
- [Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training](https://mlsanity.com/paper/2603.18112/) (2026)

## Abstract (arXiv)

While the pay-as-you-go nature of cloud virtual machines (VMs) makes it easy to spin-up large clusters for training ML models, it can also lead to ballooning costs. The 100s of virtual machine sizes provided by cloud platforms also makes it extremely challenging to select the ``right'' cloud cluster configuration for training. Furthermore, the training time and cost of distributed model training is highly sensitive to the cluster configurations, and presents a large and complex tradeoff-space. In this paper, we develop principled and practical techniques for optimizing the training time and cost of distributed ML model training on the cloud. Our key insight is that both parallel and statistical efficiency must be considered when selecting the optimum job configuration parameters such as the number of workers and the batch size. By combining conventional parallel scaling concepts and new insights into SGD noise, our models accurately estimate the time and cost on different cluster configurations with < 5% error. Using the repetitive nature of training and our models, we can search for optimum cloud configurations in a black-box, online manner. Our approach reduces training times by 2 times and costs more more than 50%. Compared to an oracle-based approach, our performance models are accurate to within 2% such that the search imposes an overhead of just 10%.