# Deep Reinforcement Agent for Scheduling in HPC

arXiv: https://arxiv.org/abs/2102.06243 · HTML: https://mlsanity.com/paper/2102.06243/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Scheduling Beyond CPUs for HPC](https://mlsanity.com/paper/2012.05439/) (2020)
- [Energy-Efficient and Thermal-Aware Resource Management for Heterogeneous Datacenters](https://mlsanity.com/paper/1410.3104/) (2014)

## Built on by (most cited first, 10 of 10)

- [DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling](https://mlsanity.com/paper/2105.07526/) (2021)
- [BFTrainer: Low-Cost Training of Neural Networks on Unfillable Supercomputer Nodes](https://mlsanity.com/paper/2106.12091/) (2021)
- [A Digital Twin Framework for Liquid-cooled Supercomputers as Demonstrated at Exascale](https://mlsanity.com/paper/2410.05133/) (2024)
- [On the impact of MDP design for Reinforcement Learning agents in Resource Management](https://mlsanity.com/paper/2109.03202/) (2021)
- [Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field](https://mlsanity.com/paper/2407.16377/) (2024)
- [Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems](https://mlsanity.com/paper/2505.03946/) (2025)
- [Hybrid Learning and Optimization-Based Dynamic Scheduling for DL Workloads on Heterogeneous GPU Clusters](https://mlsanity.com/paper/2512.10271/) (2025)
- [Attention-Informed Surrogates for Navigating Power-Performance Trade-offs in HPC](https://mlsanity.com/paper/2601.15399/) (2026)
- [What Artificial Intelligence can do for High-Performance Computing systems?](https://mlsanity.com/paper/2602.00014/) (2026)
- [Learning to Score: Tuning Cluster Schedulers through Reinforcement Learning](https://mlsanity.com/paper/2603.10545/) (2026)

## Abstract (arXiv)

Cluster scheduler is crucial in high-performance computing (HPC). It determines when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. A unique training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. The experiments with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 45%.