mlsanityData to 7 Oct 2026

Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters

arXiv 2109.01313 ↗First posted 3 Sep 2021cs.DC cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Modern GPU datacenters are critical for delivering Deep Learning (DL) models and services in both the research community and industry. When operating a datacenter, optimization of resource scheduling and management can bring significant financial benefits. Achieving this goal requires a deep understanding of the job features and user behaviors. We present a comprehensive study about the characteristics of DL jobs and resource management. First, we perform a large-scale analysis of real-world job traces from SenseTime. We uncover some interesting conclusions from the perspectives of clusters, jobs and users, which can facilitate the cluster system designs. Second, we introduce a general-purpose framework, which manages resources based on historical data. As case studies, we design: a Quasi-Shortest-Service-First scheduling service, which can minimize the cluster-wide average job completion time by up to 6.5x; and a Cluster Energy Saving service, which improves overall cluster utilization by up to 13%.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 13 of its 81 references that are in the map. First posted on arXiv 3 Sep 2021.

Builds on

13 of 81 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2019Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloadscited by 79
  6. 2020Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learningcited by 68
  7. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 65
  8. 2020Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloadscited by 56
  9. 2019Themis: Fair and Efficient GPU Cluster Schedulingcited by 43
  10. 2019Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applicationscited by 17
  11. 2019Characterizing Deep Learning Training Workloads on Alibaba-PAIcited by 11
  12. 2018A Workload Analysis of NSF's Innovative HPC Resources Using XDMoDcited by 2
1 more
  1. 2021Energy-aware Task Scheduling with Deadline Constraint in DVFS-enabled Heterogeneous Clusterscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 9; 2025: 6; 2026: 7.

31 papers.

Sort:

By area:

  1. 2024Characterization of Large Language Model Development in the Datacentercited by 46
  2. 2025ServeGen: Workload Characterization and Generation of Large Language Model Serving in Productioncited by 29
  3. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 27
  4. 2023vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Trainingcited by 22
  5. 2024Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloadscited by 17
  6. 2024Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Surveycited by 9
  7. 2026Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scalecited by 9
  8. 2023POLCA: Power Oversubscription in LLM Cloud Providerscited by 8
  9. 2022Benchmarking Resource Usage for Efficient Distributed Deep Learningcited by 7
  10. 2024A Survey on Failure Analysis and Fault Injection in AI Systemscited by 7
  11. 2022Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Visioncited by 6
  12. 2025PLoRA: Efficient Concurrent LoRA Training for Large Language Modelscited by 6
19 more
  1. 2022DNNAbacus: Toward Accurate Computational Cost Prediction for Deep Neural Networkscited by 4
  2. 2024GPU Cluster Scheduling for Network-Sensitive Deep Learningcited by 2
  3. 2024Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Surveycited by 2
  4. 2023Can Blockchains Reliably Train Machine Learning Models?cited by 1
  5. 2024Power- and Fragmentation-aware Online Scheduling for GPU Datacenterscited by 1
  6. 2025Resource Heterogeneity-Aware and Utilization-Enhanced Scheduling for Deep Learning Clusterscited by 1
  7. 2022Introducing Federated Learning into Internet of Things ecosystems -- preliminary considerationscited by 0
  8. 2023A Heavy-Load-Enhanced and Changeable-Periodicity-Perceived Workload Prediction Networkcited by 0
  9. 2023On the Burstiness of Distributed Machine Learning Trafficcited by 0
  10. 2024Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Designcited by 0
  11. 2025THOR: A Generic Energy Estimation Approach for On-Device Trainingcited by 0
  12. 2025AEGIS: Runtime-Guided GPU Collocation for Multi-Tenant Deep Learning Trainingcited by 0
  13. 2025Hybrid Learning and Optimization-Based Dynamic Scheduling for DL Workloads on Heterogeneous GPU Clusterscited by 0
  14. 2026What Artificial Intelligence can do for High-Performance Computing systems?cited by 0
  15. 2026A Physics-Aware Framework for Short-Term GPU Power Forecasting of AI Data Centerscited by 0
  16. 2026Global Simulation-Guided Dynamic Operator Scheduling for Efficient Multi-Tenant Model Servingcited by 0
  17. 2026Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verificationcited by 0
  18. 2026COMPASS-ABS: Reducing Fragmentation in Shared GPU Clusters for Deep Learning Training Workloadscited by 0
  19. 2026Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Servicescited by 0