# GPU Cluster Scheduling for Network-Sensitive Deep Learning

arXiv: https://arxiv.org/abs/2401.16492 · HTML: https://mlsanity.com/paper/2401.16492/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 102 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [MLPerf Training Benchmark](https://mlsanity.com/paper/1910.01500/) (2019)
- [Accelerating Distributed MoE Training and Inference with Lina](https://mlsanity.com/paper/2210.17223/) (2022)
- [Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads](https://mlsanity.com/paper/1901.05758/) (2019)
- [Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning](https://mlsanity.com/paper/2008.12260/) (2020)
- [ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale](https://mlsanity.com/paper/2303.14006/) (2023)
- [Themis: Fair and Efficient GPU Cluster Scheduling](https://mlsanity.com/paper/1907.01484/) (2019)
- [Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect](https://mlsanity.com/paper/1903.04611/) (2019)
- [Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters](https://mlsanity.com/paper/2109.01313/) (2021)
- [CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters](https://mlsanity.com/paper/2308.00852/) (2023)
- [Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models](https://mlsanity.com/paper/2110.04478/) (2021)
- [DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters](https://mlsanity.com/paper/1909.06040/) (2019)
- [Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters](https://mlsanity.com/paper/2108.03645/) (2021)
- [Impact of RoCE Congestion Control Policies on Distributed Training of DNNs](https://mlsanity.com/paper/2207.10898/) (2022)
- [A Sum-of-Ratios Multi-Dimensional-Knapsack Decomposition for DNN Resource Scheduling](https://mlsanity.com/paper/2105.13855/) (2021)
- [CASH: A Credit Aware Scheduling for Public Cloud Platforms](https://mlsanity.com/paper/2009.04561/) (2020)

## Built on by (most cited first, 2 of 2)

- [Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments](https://mlsanity.com/paper/2512.22149/) (2025)
- [m4: A Learned Flow-level Network Simulator](https://mlsanity.com/paper/2503.01770/) (2025)

## Abstract (arXiv)

We propose a novel GPU-cluster scheduler for distributed DL (DDL) workloads that enables proximity based consolidation of GPU resources based on the DDL jobs' sensitivities to the anticipated communication-network delays. Our scheduler consists of three major components: (i) a classical delay scheduling algorithm to facilitate job placement and consolidation; (ii) a network-sensitive job preemption strategy; and (iii) an "auto-tuner" mechanism to optimize delay timers for effective delay scheduling. Additionally, to enable a cost-effective methodology for large-scale experiments, we develop a data-driven DDL cluster simulation platform. Employing the simulation platform we compare against several state-of-the-art alternatives on real-world workload traces to demonstrate the benefits of our design. Our scheduler can provide improvement of up to 69% in end-to-end Makespan for training all jobs compared to the prevailing consolidation-based scheduling methods, while reducing the average job completion time by up to 83% and minimizing the communication overheads by up to 98% under congested networking conditions.