GPU Cluster Scheduling for Network-Sensitive Deep Learning
Not on the map. Why?
Abstract (arXiv)
We propose a novel GPU-cluster scheduler for distributed DL (DDL) workloads that enables proximity based consolidation of GPU resources based on the DDL jobs' sensitivities to the anticipated communication-network delays. Our scheduler consists of three major components: (i) a classical delay scheduling algorithm to facilitate job placement and consolidation; (ii) a network-sensitive job preemption strategy; and (iii) an "auto-tuner" mechanism to optimize delay timers for effective delay scheduling. Additionally, to enable a cost-effective methodology for large-scale experiments, we develop a data-driven DDL cluster simulation platform. Employing the simulation platform we compare against several state-of-the-art alternatives on real-world workload traces to demonstrate the benefits of our design. Our scheduler can provide improvement of up to 69% in end-to-end Makespan for training all jobs compared to the prevailing consolidation-based scheduling methods, while reducing the average job completion time by up to 83% and minimizing the communication overheads by up to 98% under congested networking conditions.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 102 references that are in the map. First posted on arXiv 29 Jan 2024.
Builds on
21 of 102 references.
References · 21
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2016Identity Mappings in Deep Residual Networkscited by 1,965
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
- 2019MLPerf Training Benchmarkcited by 115
- 2022Accelerating Distributed MoE Training and Inference with Linacited by 109
- 2019Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloadscited by 79
- 2020Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learningcited by 72
- 2023ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scalecited by 47
- 2019Themis: Fair and Efficient GPU Cluster Schedulingcited by 43
9 more
- 2019Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirectcited by 41
- 2021Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenterscited by 31
- 2023CASSINI: Network-Aware Job Scheduling in Machine Learning Clusterscited by 26
- 2021Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Modelscited by 16
- 2019DL2: A Deep Learning-driven Scheduler for Deep Learning Clusterscited by 14
- 2021Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusterscited by 7
- 2022Impact of RoCE Congestion Control Policies on Distributed Training of DNNscited by 6
- 2021A Sum-of-Ratios Multi-Dimensional-Knapsack Decomposition for DNN Resource Schedulingcited by 5
- 2020CASH: A Credit Aware Scheduling for Public Cloud Platformscited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.