# Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approach

arXiv: https://arxiv.org/abs/2309.07265 · HTML: https://mlsanity.com/paper/2309.07265/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Successor Features for Transfer in Reinforcement Learning](https://mlsanity.com/paper/1606.05312/) (2016)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial](https://mlsanity.com/paper/2011.03615/) (2020)
- [Intelligence and Learning in O-RAN for Data-driven NextG Cellular Networks](https://mlsanity.com/paper/2012.01263/) (2020)
- [ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms](https://mlsanity.com/paper/2112.09559/) (2021)
- [Deep Reinforcement Learning for Resource Management in Network Slicing](https://mlsanity.com/paper/1805.06591/) (2018)
- [Experienced Deep Reinforcement Learning with Generative Adversarial Networks (GANs) for Model-Free Ultra Reliable Low Latency Communication](https://mlsanity.com/paper/1911.03264/) (2019)
- [Transfer Learning for Future Wireless Networks: A Comprehensive Survey](https://mlsanity.com/paper/2102.07572/) (2021)
- [Toward Next Generation Open Radio Access Network--What O-RAN Can and Cannot Do!](https://mlsanity.com/paper/2111.13754/) (2021)
- [RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN](https://mlsanity.com/paper/2111.06978/) (2021)
- [Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing](https://mlsanity.com/paper/2208.01736/) (2022)
- [Communication and Computation O-RAN Resource Slicing for URLLC Services Using Deep Reinforcement Learning](https://mlsanity.com/paper/2202.06439/) (2022)
- [Federated Deep Reinforcement Learning for Open RAN Slicing in 6G Networks](https://mlsanity.com/paper/2206.11328/) (2022)
- [Evolutionary Deep Reinforcement Learning for Dynamic Slice Management in O-RAN](https://mlsanity.com/paper/2208.14394/) (2022)
- [Virtual Reality Gaming on the Cloud: A Reality Check](https://mlsanity.com/paper/2109.10114/) (2021)
- [Artificial Intelligence in Open Radio Access Network](https://mlsanity.com/paper/2104.09445/) (2021)
- [Toward Safe and Accelerated Deep Reinforcement Learning for Next-Generation Wireless Networks](https://mlsanity.com/paper/2209.13532/) (2022)
- [Knowledge Transfer and Reuse: A Case Study of AI-enabled Resource Management in RAN Slicing](https://mlsanity.com/paper/2212.09172/) (2022)
- [Intelligent Closed-loop RAN Control with xApps in OpenRAN Gym](https://mlsanity.com/paper/2208.14877/) (2022)
- [How Does Forecasting Affect the Convergence of DRL Techniques in O-RAN Slicing?](https://mlsanity.com/paper/2309.00489/) (2023)

## Built on by (most cited first, 5 of 5)

- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Near-Real-Time Resource Slicing for QoS Optimization in 5G O-RAN using Deep Reinforcement Learning](https://mlsanity.com/paper/2509.14343/) (2025)
- [ML-Enabled Open RAN: A Comprehensive Survey of Architectures, Challenges, and Opportunities](https://mlsanity.com/paper/2604.01239/) (2026)
- [Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06968/) (2025)
- [Bayesian Optimization for Non-Cooperative Game-Based Radio Resource Management](https://mlsanity.com/paper/2512.01245/) (2025)

## Abstract (arXiv)

The open radio access network (O-RAN) architecture supports intelligent network control algorithms as one of its core capabilities. Data-driven applications incorporate such algorithms to optimize radio access network (RAN) functions via RAN intelligent controllers (RICs). Deep reinforcement learning (DRL) algorithms are among the main approaches adopted in the O-RAN literature to solve dynamic radio resource management problems. However, despite the benefits introduced by the O-RAN RICs, the practical adoption of DRL algorithms in real network deployments falls behind. This is primarily due to the slow convergence and unstable performance exhibited by DRL agents upon deployment and when encountering previously unseen network conditions. In this paper, we address these challenges by proposing transfer learning (TL) as a core component of the training and deployment workflows for the DRL-based closed-loop control of O-RAN functionalities. To this end, we propose and design a hybrid TL-aided approach that leverages the advantages of both policy reuse and distillation TL methods to provide safe and accelerated convergence in DRL-based O-RAN slicing. We conduct a thorough experiment that accommodates multiple services, including real VR gaming traffic to reflect practical scenarios of O-RAN slicing. We also propose and implement policy reuse and distillation-aided DRL and non-TL-aided DRL as three separate baselines. The proposed hybrid approach shows at least: 7.7% and 20.7% improvements in the average initial reward value and the percentage of converged scenarios, and a 64.6% decrease in reward variance while maintaining fast convergence and enhancing the generalizability compared with the baselines.