# Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization

arXiv: https://arxiv.org/abs/2509.22701 · HTML: https://mlsanity.com/paper/2509.22701/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 35 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 5 of 5)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [The Case for Task Sampling based Learning for Cluster Job Scheduling](https://mlsanity.com/paper/2108.10464/) (2021)
- [DLB: Deep Learning Based Load Balancing](https://mlsanity.com/paper/1910.08494/) (2019)
- [Network Contention-Aware Cluster Scheduling with Reinforcement Learning](https://mlsanity.com/paper/2310.20209/) (2023)

## Abstract (arXiv)

This study presents a machine learning-assisted approach to optimize task scheduling in cluster systems, focusing on node-affinity constraints. Traditional schedulers like Kubernetes struggle with real-time adaptability, whereas the proposed continuous transfer learning model evolves dynamically during operations, minimizing retraining needs. Evaluated on Google Cluster Data, the model achieves over 99% accuracy, reducing computational overhead and improving scheduling latency for constrained tasks. This scalable solution enables real-time optimization, advancing machine learning integration in cluster management and paving the way for future adaptive scheduling strategies.