# Learning to Score: Tuning Cluster Schedulers through Reinforcement Learning

arXiv: https://arxiv.org/abs/2603.10545 · HTML: https://mlsanity.com/paper/2603.10545/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Hyp-RL : Hyperparameter Optimization by Reinforcement Learning](https://mlsanity.com/paper/1906.11527/) (2019)
- [Deep Reinforcement Agent for Scheduling in HPC](https://mlsanity.com/paper/2102.06243/) (2021)
- [Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimization](https://mlsanity.com/paper/2205.11104/) (2022)
- [Characterizing Co-located Datacenter Workloads: An Alibaba Case Study](https://mlsanity.com/paper/1808.02919/) (2018)
- [High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB](https://mlsanity.com/paper/2103.16267/) (2021)
- [A Framework for Automated Cellular Network Tuning with Reinforcement Learning](https://mlsanity.com/paper/1808.05140/) (2018)
- [Magpie: Automatically Tuning Static Parameters for Distributed File Systems using Deep Reinforcement Learning](https://mlsanity.com/paper/2207.09298/) (2022)

## Abstract (arXiv)

Efficiently allocating incoming jobs to nodes in large-scale clusters can lead to substantial improvements in both cluster utilization and job performance. In order to allocate incoming jobs, cluster schedulers usually rely on a set of scoring functions to rank feasible nodes. Results from individual scoring functions are usually weighted equally, which could lead to sub-optimal deployments as the one-size-fits-all solution does not take into account the characteristics of each workload. Tuning the weights of scoring functions, however, requires expert knowledge and is computationally expensive. This paper proposes a reinforcement learning approach for learning the weights in scheduler scoring algorithms with the overall objective of improving the end-to-end performance of jobs for a given cluster. Our approach is based on percentage improvement reward, frame-stacking, and limiting domain information. We propose a percentage improvement reward to address the objective of multi-step parameter tuning. The inclusion of frame-stacking allows for carrying information across an optimization experiment. Limiting domain information prevents overfitting and improves performance in unseen clusters and workloads. The policy is trained on different combinations of workloads and cluster setups. We demonstrate the proposed approach improves performance on average by 33% compared to fixed weights and 12% compared to the best-performing baseline in a lab-based serverless scenario.