# Advancing RAN Slicing with Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.10547 · HTML: https://mlsanity.com/paper/2312.10547/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Placed in

- Offline RL and imitation learning from data / [Offline multi-agent RL](https://mlsanity.com/offlinerl/offline-multi-agent-rl/)

## Builds on (most cited first, 22 of 22)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble](https://mlsanity.com/paper/2110.01548/) (2021)
- [Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism](https://mlsanity.com/paper/2103.12021/) (2021)
- [Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning](https://mlsanity.com/paper/2106.04895/) (2021)
- [When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?](https://mlsanity.com/paper/2204.05618/) (2022)
- [Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game](https://mlsanity.com/paper/2205.15512/) (2022)
- [Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2002.06215/) (2020)
- [Towards Optimal Power Control via Ensembling Deep Neural Networks](https://mlsanity.com/paper/1807.10025/) (2018)
- [ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms](https://mlsanity.com/paper/2112.09559/) (2021)
- [Experienced Deep Reinforcement Learning with Generative Adversarial Networks (GANs) for Model-Free Ultra Reliable Low Latency Communication](https://mlsanity.com/paper/1911.03264/) (2019)
- [Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets](https://mlsanity.com/paper/2311.11423/) (2023)
- [A Deep Q-Learning Method for Downlink Power Allocation in Multi-Cell Networks](https://mlsanity.com/paper/1904.13032/) (2019)
- [RAN Resource Slicing in 5G Using Multi-Agent Correlated Q-Learning](https://mlsanity.com/paper/2107.01018/) (2021)
- [OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning](https://mlsanity.com/paper/2111.01616/) (2021)
- [Inter-Cell Slicing Resource Partitioning via Coordinated Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2202.12833/) (2022)
- [Near-optimal Conservative Exploration in Reinforcement Learning under Episode-wise Constraints](https://mlsanity.com/paper/2306.06265/) (2023)
- [Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources](https://mlsanity.com/paper/2306.08364/) (2023)
- [Small Cell Transmit Power Assignment Based on Correlated Bandit Learning](https://mlsanity.com/paper/1703.05975/) (2017)

## Built on by (most cited first, 5 of 5)

- [Average Reward Reinforcement Learning for Wireless Radio Resource Management](https://mlsanity.com/paper/2501.06700/) (2025)
- [Learning to Slice Wi-Fi Networks: A State-Augmented Primal-Dual Approach](https://mlsanity.com/paper/2405.05748/) (2024)
- [Decision Feedback In-Context Learning for Wireless Symbol Detection](https://mlsanity.com/paper/2503.16594/) (2025)
- [Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control](https://mlsanity.com/paper/2603.03932/) (2026)
- [Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management](https://mlsanity.com/paper/2608.28878/) (2026)

## Abstract (arXiv)

Dynamic radio resource management (RRM) in wireless networks presents significant challenges, particularly in the context of Radio Access Network (RAN) slicing. This technology, crucial for catering to varying user requirements, often grapples with complex optimization scenarios. Existing Reinforcement Learning (RL) approaches, while achieving good performance in RAN slicing, typically rely on online algorithms or behavior cloning. These methods necessitate either continuous environmental interactions or access to high-quality datasets, hindering their practical deployment. Towards addressing these limitations, this paper introduces offline RL to solving the RAN slicing problem, marking a significant shift towards more feasible and adaptive RRM methods. We demonstrate how offline RL can effectively learn near-optimal policies from sub-optimal datasets, a notable advancement over existing practices. Our research highlights the inherent flexibility of offline RL, showcasing its ability to adjust policy criteria without the need for additional environmental interactions. Furthermore, we present empirical evidence of the efficacy of offline RL in adapting to various service-level requirements, illustrating its potential in diverse RAN slicing scenarios.