mlsanityData to 14 Oct 2026

Average Reward Reinforcement Learning for Wireless Radio Resource Management

arXiv 2501.06700 ↗First posted 12 Jan 2025cs.IT cs.LG cs.NIcited by 3

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

In this paper, we address a crucial but often overlooked issue in applying reinforcement learning (RL) to radio resource management (RRM) in wireless communications: the mismatch between the discounted reward RL formulation and the undiscounted goal of wireless network optimization. To the best of our knowledge, we are the first to systematically investigate this discrepancy, starting with a discussion of the problem formulation followed by simulations that quantify the extent of the gap. To bridge this gap, we introduce the use of average reward RL, a method that aligns more closely with the long-term objectives of RRM. We propose a new method called the Average Reward Off policy Soft Actor Critic (ARO SAC) is an adaptation of the well known Soft Actor Critic algorithm in the average reward framework. This new method achieves significant performance improvement our simulation results demonstrate a 15% gain in the system performance over the traditional discounted reward RL approach, underscoring the potential of average reward RL in enhancing the efficiency and effectiveness of wireless network optimization.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 34 references that are in the map. First posted on arXiv 12 Jan 2025.

Builds on

18 of 34 references.

References · 18

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 126
  3. 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
  4. 2022Supported Policy Optimization for Offline Reinforcement Learningcited by 74
  5. 2018Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networkscited by 70
  6. 2020Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learningcited by 42
  7. 2021ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platformscited by 35
  8. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 31
  9. 2020Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learningcited by 28
  10. 2021Average-Reward Reinforcement Learning with Trust Region Methodscited by 19
  11. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasetscited by 12
  12. 2020A Deeper Look at Discounting Mismatch in Actor-Critic Algorithmscited by 10
6 more
  1. 2023Off-Policy Average Reward Actor-Critic with Deterministic Policy Searchcited by 10
  2. 2019A Deep Q-Learning Method for Downlink Power Allocation in Multi-Cell Networkscited by 7
  3. 2020Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networkscited by 7
  4. 2018Deep Learning for Radio Resource Allocation in Multi-Cell Networkscited by 6
  5. 2020Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networkscited by 5
  6. 2023Advancing RAN Slicing with Offline Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Almost Sure Convergence of Differential Temporal Difference Learning for Average Reward Markov Decision Processescited by 2
  2. 2025Decision Feedback In-Context Learning for Wireless Symbol Detectioncited by 0
  3. 2026Planning or Learning: Reliability and Cost in Multi-Asset Maintenancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Offline and Distributional Reinforcement Learning for Radio Resource Management
  2. 2025An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management
  3. 2026Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization
  4. 2025Offline and Distributional Reinforcement Learning for Wireless Communications
  5. 2026Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints
  6. 2025Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models
  7. 2023Communication Load Balancing via Efficient Inverse Reinforcement Learning
  8. 2026Large Language Model (LLM)-enabled Reinforcement Learning for Wireless Network Optimization