Dynamics of Resource Allocation in O-RANs: An In-depth Exploration of On-Policy and Off-Policy Deep Reinforcement Learning for Real-Time Applications
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Deep Reinforcement Learning (DRL) is a powerful tool used for addressing complex challenges in mobile networks. This paper investigates the application of two DRL models, on-policy and off-policy, in the field of resource allocation for Open Radio Access Networks (O-RAN). The on-policy model is the Proximal Policy Optimization (PPO), and the off-policy model is the Sample Efficient Actor-Critic with Experience Replay (ACER), which focuses on resolving the challenges of resource allocation associated with a Quality of Service (QoS) application that has strict requirements. Motivated by the original work of Nessrine Hammami and Kim Khoa Nguyen, this study is a replication to validate and prove the findings. Both PPO and ACER are used within the same experimental setup to assess their performance in a scenario of latency-sensitive and latency-tolerant users and compare them. The aim is to verify the efficacy of on-policy and off-policy DRL models in the context of O-RAN resource allocation. Results from this replication contribute to the ongoing scientific research and offer insights into the reproducibility and generalizability of the original research. This analysis reaffirms that both on-policy and off-policy DRL models have better performance than greedy algorithms in O-RAN settings. In addition, it confirms the original observations that the on-policy model (PPO) gives a favorable balance between energy consumption and user latency, while the off-policy model (ACER) shows a faster convergence. These findings give good insights to optimize resource allocation strategies in O-RANs. Index Terms: 5G, O-RAN, resource allocation, ML, DRL, PPO, ACER.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 16 references that are in the map. First posted on arXiv 17 Nov 2024.
Builds on
5 of 16 references.
References · 5
- 2016Sample Efficient Actor-Critic with Experience Replaycited by 188
- 2019Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansioncited by 13
- 2023Intelligent Load Balancing and Resource Allocation in O-RAN: A Multi-Agent Multi-Armed Bandit Approachcited by 2
- 2022Deep Reinforcement Learning-based Radio Resource Allocation and Beam Management under Location Uncertainty in 5G mmWave Networkscited by 1
- 2022Reinforcement learning with experience replay and adaptation of action dispersioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Near-Real-Time Resource Slicing for QoS Optimization in 5G O-RAN using Deep Reinforcement Learning
- 2024Meta Reinforcement Learning Approach for Adaptive Resource Optimization in O-RAN
- 2025Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless Networks
- 2025Optimisation of Resource Allocation in Heterogeneous Wireless Networks Using Deep Reinforcement Learning
- 2025QoS-Aware Dynamic CU Selection in O-RAN with Graph-Based Reinforcement Learning
- 2023Advancing RAN Slicing with Offline Reinforcement Learning
- 2025Percentile-Based Deep Reinforcement Learning and Reward Based Personalization For Delay Aware RAN Slicing in O-RAN
- 2025Towards Scalable O-RAN Resource Management: Graph-Augmented Proximal Policy Optimization