# An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management

arXiv: https://arxiv.org/abs/2501.12991 · HTML: https://mlsanity.com/paper/2501.12991/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Placed in

- Offline RL and imitation learning from data / [Offline multi-agent RL](https://mlsanity.com/offlinerl/offline-multi-agent-rl/)

## Builds on (most cited first, 23 of 23)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Value-Decomposition Multi-Agent Actor-Critics](https://mlsanity.com/paper/2007.12306/) (2020)
- [Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2002.06215/) (2020)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Learning Resilient Radio Resource Management Policies with Graph Neural Networks](https://mlsanity.com/paper/2203.11012/) (2022)
- [Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets](https://mlsanity.com/paper/2311.11423/) (2023)
- [Multi-agent Reinforcement Learning for Dynamic Resource Management in 6G in-X Subnetworks](https://mlsanity.com/paper/2205.05036/) (2022)
- [Multi-UAV Path Learning for Age and Power Optimization in IoT with UAV Battery Recharge](https://mlsanity.com/paper/2301.03423/) (2023)
- [Traffic Prediction and Fast Uplink for Hidden Markov IoT Models](https://mlsanity.com/paper/2209.05939/) (2022)
- [Age Minimization in Massive IoT via UAV Swarm: A Multi-agent Reinforcement Learning Approach](https://mlsanity.com/paper/2309.14757/) (2023)
- [Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models](https://mlsanity.com/paper/2401.13827/) (2024)
- [Offline and Distributional Reinforcement Learning for Radio Resource Management](https://mlsanity.com/paper/2409.16764/) (2024)
- [Game-theoretic Resource Allocation Methods for Device-to-Device (D2D) Communication](https://mlsanity.com/paper/1403.5723/) (2014)

## Built on by (most cited first, 5 of 5)

- [Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning](https://mlsanity.com/paper/2502.01268/) (2025)
- [Federated Multi-Agent Reinforcement Learning for Privacy-Preserving and Energy-Aware Resource Management in 6G Edge Networks](https://mlsanity.com/paper/2509.10163/) (2025)
- [Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions](https://mlsanity.com/paper/2601.00321/) (2026)
- [Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints](https://mlsanity.com/paper/2608.01745/) (2026)
- [Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management](https://mlsanity.com/paper/2608.28878/) (2026)

## Abstract (arXiv)

Offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL, such as safety concerns, expensive data collection, extended training intervals, and high signaling overhead caused by online interactions with the environment. In this work, we propose an offline MARL algorithm for radio resource management (RRM), focusing on optimizing scheduling policies for multiple access points (APs) to jointly maximize the sum and tail rates of user equipment (UEs). We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE). Our simulation results demonstrate that the proposed offline MARL framework outperforms conventional baseline approaches, achieving over a 15% improvement in a weighted combination of sum and tail rates. Additionally, the CTDE framework strikes an effective balance, reducing the computational complexity of centralized methods while addressing the inefficiencies of independent training. These results underscore the potential of offline MARL to deliver scalable, robust, and efficient solutions for resource management in dynamic wireless networks.