# From Simulation to Practice: Generalizable Deep Reinforcement Learning for Cellular Schedulers

arXiv: https://arxiv.org/abs/2411.08529 · HTML: https://mlsanity.com/paper/2411.08529/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2024.

## Built on by (most cited first, 2 of 2)

- [Enhancing User Throughput in Multi-panel mmWave Radio Access Networks for Beam-based MU-MIMO Using a DRL Method](https://mlsanity.com/paper/2603.02745/) (2026)
- [Combining Improvements in Uplink AI-RAN](https://mlsanity.com/paper/2610.05936/) (2026)

## Abstract (arXiv)

Efficient radio packet scheduling remains one of the most challenging tasks in cellular networks, and while heuristic methods exist, practical deep learning-based schedulers that are 3GPP-compliant and capable of real-time operation in 5G and beyond are still missing. To address this, we first take a critical look at previous deep scheduler efforts. Secondly, we enhance State-of-the-Art (SoTA) deep Reinforcement Learning (RL) algorithms and adapt them to train our deep scheduler. In particular, we propose a novel combination of training techniques for Proximal Policy Optimization (PPO) and a new Distributional Soft Actor-Critic Discrete (DSACD) algorithm, which outperformed other variants tested. These improvements were achieved while maintaining minimal actor network complexity, making them suitable for real-time computing environments. Furthermore, entropy learning in SACD was fine-tuned to accommodate resource allocation action spaces of varying sizes. Our proposed deep schedulers exhibited strong generalization across different bandwidths, number of Multi-User MIMO (MU-MIMO) layers, and traffic models. Ultimately, we show that our pre-trained deep schedulers outperform their heuristic rivals in realistic and standard-compliant 5G system-level simulations.