# Average Reward Reinforcement Learning for Wireless Radio Resource Management

arXiv: https://arxiv.org/abs/2501.06700 · HTML: https://mlsanity.com/paper/2501.06700/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 12 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 18 of 18)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Revisiting the Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2305.09836/) (2023)
- [Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01929/) (2016)
- [Supported Policy Optimization for Offline Reinforcement Learning](https://mlsanity.com/paper/2202.06239/) (2022)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2002.06215/) (2020)
- [ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms](https://mlsanity.com/paper/2112.09559/) (2021)
- [Average-Reward Off-Policy Policy Evaluation with Function Approximation](https://mlsanity.com/paper/2101.02808/) (2021)
- [Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning](https://mlsanity.com/paper/2004.10888/) (2020)
- [Average-Reward Reinforcement Learning with Trust Region Methods](https://mlsanity.com/paper/2106.03442/) (2021)
- [Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets](https://mlsanity.com/paper/2311.11423/) (2023)
- [A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms](https://mlsanity.com/paper/2010.01069/) (2020)
- [Off-Policy Average Reward Actor-Critic with Deterministic Policy Search](https://mlsanity.com/paper/2305.12239/) (2023)
- [A Deep Q-Learning Method for Downlink Power Allocation in Multi-Cell Networks](https://mlsanity.com/paper/1904.13032/) (2019)
- [Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks](https://mlsanity.com/paper/2009.06681/) (2020)
- [Deep Learning for Radio Resource Allocation in Multi-Cell Networks](https://mlsanity.com/paper/1808.00667/) (2018)
- [Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks](https://mlsanity.com/paper/2012.10682/) (2020)
- [Advancing RAN Slicing with Offline Reinforcement Learning](https://mlsanity.com/paper/2312.10547/) (2023)

## Built on by (most cited first, 3 of 3)

- [Almost Sure Convergence of Differential Temporal Difference Learning for Average Reward Markov Decision Processes](https://mlsanity.com/paper/2602.16629/) (2026)
- [Decision Feedback In-Context Learning for Wireless Symbol Detection](https://mlsanity.com/paper/2503.16594/) (2025)
- [Planning or Learning: Reliability and Cost in Multi-Asset Maintenance](https://mlsanity.com/paper/2609.13566/) (2026)

## Abstract (arXiv)

In this paper, we address a crucial but often overlooked issue in applying reinforcement learning (RL) to radio resource management (RRM) in wireless communications: the mismatch between the discounted reward RL formulation and the undiscounted goal of wireless network optimization. To the best of our knowledge, we are the first to systematically investigate this discrepancy, starting with a discussion of the problem formulation followed by simulations that quantify the extent of the gap. To bridge this gap, we introduce the use of average reward RL, a method that aligns more closely with the long-term objectives of RRM. We propose a new method called the Average Reward Off policy Soft Actor Critic (ARO SAC) is an adaptation of the well known Soft Actor Critic algorithm in the average reward framework. This new method achieves significant performance improvement our simulation results demonstrate a 15% gain in the system performance over the traditional discounted reward RL approach, underscoring the potential of average reward RL in enhancing the efficiency and effectiveness of wireless network optimization.