# Srifty: Swift and Thrifty Distributed Training on the Cloud

arXiv: https://arxiv.org/abs/2011.14243 · HTML: https://mlsanity.com/paper/2011.14243/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 4 of 4)

- [M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity](https://mlsanity.com/paper/2404.14527/) (2024)
- [Efficient Direct-Connect Topologies for Collective Communications](https://mlsanity.com/paper/2202.03356/) (2022)
- [Echo: Simulating Distributed Training At Scale](https://mlsanity.com/paper/2412.12487/) (2024)
- [FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost](https://mlsanity.com/paper/2604.24073/) (2026)

## Abstract (arXiv)

Finding the best VM configuration is key to achieve lower cost and higher throughput, two primary concerns in cloud-based distributed neural network (NN) training today. Optimal VM selection that meets user constraints requires efficiently navigating a large search space while controlling for the performance variance associated with sharing cloud instances and networks. In this work, we characterize this variance in the context of distributed NN training and present results of a comprehensive throughput and cost-efficiency study we conducted across a wide array of instances to prune for the optimal VM search space. Using insights from these studies, we built Srifty, a system that combines runtime profiling with learned performance models to accurately predict training performance and find the best VM choice that satisfies user constraints, potentially leveraging both heterogeneous setups and spot instances. We integrated Srifty with PyTorch and evaluated it on Amazon EC2. We conducted a large-scale generalization study of Srifty across more than 2K training setups on EC2. Our results show that Srifty achieves an iteration latency prediction error of 8%, and its VM instance recommendations offer significant throughput gain and cost reduction while satisfying user constraints compared to existing solutions in complex, real-world scenarios.