mlsanityData to 14 Oct 2026

Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networks

arXiv 2404.16920 ↗First posted 25 Apr 2024cs.NI cs.IT cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

We study the data packet transmission problem (mmDPT) in dense cell-free millimeter wave (mmWave) networks, i.e., users sending data packet requests to access points (APs) via uplinks and APs transmitting requested data packets to users via downlinks. Our objective is to minimize the average delay in the system due to APs' limited service capacity and unreliable wireless channels between APs and users. This problem can be formulated as a restless multi-armed bandits problem with fairness constraint (RMAB-F). Since finding the optimal policy for RMAB-F is intractable, existing learning algorithms are computationally expensive and not suitable for practical dynamic dense mmWave networks. In this paper, we propose a structured reinforcement learning (RL) solution for mmDPT by exploiting the inherent structure encoded in RMAB-F. To achieve this, we first design a low-complexity and provably asymptotically optimal index policy for RMAB-F. Then, we leverage this structure information to develop a structured RL algorithm called mmDPT-TS, which provably achieves an tildeO(√T) Bayesian regret. More importantly, mmDPT-TS is computation-efficient and thus amenable to practical implementation, as it fully exploits the structure of index policy for making decisions. Extensive emulation based on data collected in realistic mmWave networks demonstrate significant gains of mmDPT-TS over existing approaches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 59 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

19 of 59 references.

References · 19

  1. 2019Combinatorial Sleeping Bandits with Fairness Constraintscited by 68
  2. 2017Learning Unknown Markov Decision Processes: A Thompson Sampling Approachcited by 65
  3. 2020Whittle index based Q-learning for restless bandits with average rewardcited by 45
  4. 2019Regret Bounds for Thompson Sampling in Episodic Restless Bandit Problemscited by 34
  5. 2020Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Banditscited by 27
  6. 2014What Will 5G Be?cited by 25
  7. 2021Q-Learning Lagrange Policies for Multi-Action Restless Banditscited by 21
  8. 2017An Asymptotically Optimal Index Policy for Finite-Horizon Restless Banditscited by 20
  9. 2021Restless Bandits with Many Arms: Beating the Central Limit Theoremcited by 18
  10. 2017End-to-End Simulation of 5G mmWave Networkscited by 10
  11. 2022On learning Whittle index policy for restless bandits with scalable regretcited by 10
  12. 2023Dynamic priority allocation via restless bandit marginal productivity indicescited by 8
7 more
  1. 2015User Association in 5G Networks: A Survey and an Outlookcited by 7
  2. 2021User Association in Dense mmWave Networks as Restless Banditscited by 5
  3. 2021A Reinforcement Learning Approach for Scheduling in mmWave Networkscited by 3
  4. 2017Low-Latency Millimeter-Wave Communications: Traffic Dispersion or Network Densification?cited by 2
  5. 2017Whittle Indexability in Egalitarian Processor Sharing Systemscited by 1
  6. 2020Delay-Aware Scheduling over mmWave/Sub-6 Dual Interfaces: A Reinforcement Learning Approachcited by 1
  7. 2021A Q-Learning-based Approach for Distributed Beam Scheduling in mmWave Networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Delay-Optimal Transmission Scheduling Policies for Time-Correlated Fading Channels ↗
  2. 2026Enhancing User Throughput in Multi-panel mmWave Radio Access Networks for Beam-based MU-MIMO Using a DRL Method
  3. 2021Reinforcement Learning for Beam Pattern Design in Millimeter Wave and Massive MIMO Systems
  4. 2026Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage
  5. 2025Meta-Reinforcement Learning for Fast and Data-Efficient Spectrum Allocation in Dynamic Wireless Networks
  6. 2025Energy Efficient Sleep Mode Optimization in 5G mmWave Networks via Multi Agent Deep Reinforcement Learning
  7. 2024Optimizing Wireless Discontinuous Reception via MAC Signaling Learning
  8. 2026Learning to Reflect: Hierarchical Multi-Agent Reinforcement Learning for CSI-Free mmWave Beam-Focusing