# Themis: Fair and Efficient GPU Cluster Scheduling

arXiv: https://arxiv.org/abs/1907.01484 · HTML: https://mlsanity.com/paper/1907.01484/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2019.

## Built on by (most cited first, 30 of 43)

- [Pathways: Asynchronous Distributed Dataflow for ML](https://mlsanity.com/paper/2203.12533/) (2022)
- [Fairness in Serving Large Language Models](https://mlsanity.com/paper/2401.00588/) (2023)
- [Efficiently Scaling LLM Reasoning with Certaindex](https://mlsanity.com/paper/2412.20993/) (2024)
- [Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs](https://mlsanity.com/paper/2204.12013/) (2022)
- [Characterization of Large Language Model Development in the Datacenter](https://mlsanity.com/paper/2403.07648/) (2024)
- [Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters](https://mlsanity.com/paper/2109.01313/) (2021)
- [CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters](https://mlsanity.com/paper/2308.00852/) (2023)
- [vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Training](https://mlsanity.com/paper/2312.12391/) (2023)
- [Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing](https://mlsanity.com/paper/2201.06227/) (2022)
- [ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation](https://mlsanity.com/paper/2405.14009/) (2024)
- [Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads](https://mlsanity.com/paper/2202.07848/) (2022)
- [Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem](https://mlsanity.com/paper/2109.11067/) (2021)
- [Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling](https://mlsanity.com/paper/2409.03365/) (2024)
- [InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers](https://mlsanity.com/paper/2502.03885/) (2025)
- [Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision](https://mlsanity.com/paper/2205.11913/) (2022)
- [PLoRA: Efficient Concurrent LoRA Training for Large Language Models](https://mlsanity.com/paper/2508.02932/) (2025)
- [SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters](https://mlsanity.com/paper/2605.00528/) (2026)
- [Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference](https://mlsanity.com/paper/2508.19559/) (2025)
- [Efficient Strong Scaling Through Burst Parallel Training](https://mlsanity.com/paper/2112.10065/) (2021)
- [Ensuring Fair LLM Serving Amid Diverse Applications](https://mlsanity.com/paper/2411.15997/) (2024)
- [Robust Heuristic Algorithm Design with LLMs](https://mlsanity.com/paper/2510.08755/) (2025)
- [Saturn: An Optimized Data System for Large Model Deep Learning Workloads](https://mlsanity.com/paper/2309.01226/) (2023)
- [GPU Cluster Scheduling for Network-Sensitive Deep Learning](https://mlsanity.com/paper/2401.16492/) (2024)
- [EdgeOL: Efficient in-situ Online Learning on Edge Devices](https://mlsanity.com/paper/2401.16694/) (2024)
- [Enabling Level-4 Autonomous Driving on a Single 1k Off-the-Shelf Card](https://mlsanity.com/paper/2110.06373/) (2021)
- [Scheduling Virtual Conferences Fairly: Achieving Equitable Participant and Speaker Satisfaction](https://mlsanity.com/paper/2204.12062/) (2022)
- [Network Contention-Aware Cluster Scheduling with Reinforcement Learning](https://mlsanity.com/paper/2310.20209/) (2023)
- [The Price of Anarchy in Disaggregated Inference](https://mlsanity.com/paper/2606.17081/) (2026)
- [Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters](https://mlsanity.com/paper/2110.06073/) (2021)
- [Deep Reinforcement Learning for System-on-Chip: Myths and Realities](https://mlsanity.com/paper/2207.14595/) (2022)

## Abstract (arXiv)

Modern distributed machine learning (ML) training workloads benefit significantly from leveraging GPUs. However, significant contention ensues when multiple such workloads are run atop a shared cluster of GPUs. A key question is how to fairly apportion GPUs across workloads. We find that established cluster scheduling disciplines are a poor fit because of ML workloads' unique attributes: ML jobs have long-running tasks that need to be gang-scheduled, and their performance is sensitive to tasks' relative placement. We propose Themis, a new scheduling framework for ML training workloads. It's GPU allocation policy enforces that ML workloads complete in a finish-time fair manner, a new notion we introduce. To capture placement sensitivity and ensure efficiency, Themis uses a two-level scheduling architecture where ML workloads bid on available resources that are offered in an auction run by a central arbiter. Our auction design allocates GPUs to winning bids by trading off efficiency for fairness in the short term but ensuring finish-time fairness in the long term. Our evaluation on a production trace shows that Themis can improve fairness by more than 2.25X and is ~5% to 250% more cluster efficient in comparison to state-of-the-art schedulers.