Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report
Not on the map. Why?
Abstract (arXiv)
Deep Reinforcement Learning (DRL) offers a powerful approach to training neural network control policies for stochastic queuing networks (SQN). However, traditional DRL methods rely on offline simulations or static datasets, limiting their real-world application in SQN control. This work proposes Online Deep Reinforcement Learning-based Controls (ODRLC) as an alternative, where an intelligent agent interacts directly with a real environment and learns an optimal control policy from these online interactions. SQNs present a challenge for ODRLC due to the unbounded nature of the queues within the network resulting in an unbounded state-space. An unbounded state-space is particularly challenging for neural network policies as neural networks are notoriously poor at extrapolating to unseen states. To address this challenge, we propose an intervention-assisted framework that leverages strategic interventions from known stable policies to ensure the queue sizes remain bounded. This framework combines the learning power of neural networks with the guaranteed stability of classical control policies for SQNs. We introduce a method to design these intervention-assisted policies to ensure strong stability of the network. Furthermore, we extend foundational DRL theorems for intervention-assisted policies and develop two practical algorithms specifically for ODRLC of SQNs. Finally, we demonstrate through experiments that our proposed algorithms outperform both classical control approaches and prior ODRLC algorithms.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 5 Apr 2024.
Builds on
17 of 31 references.
References · 17
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
- 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
- 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
- 2020Projection-Based Constrained Policy Optimizationcited by 195
- 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
- 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
- 2018Generalization and Regularization in DQNcited by 117
- 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 86
- 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
- 2020Queueing Network Controls via Deep Reinforcement Learningcited by 40
5 more
- 2021On-Policy Deep Reinforcement Learning for the Average-Reward Criterioncited by 29
- 2020Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamicscited by 20
- 2021Average-Reward Reinforcement Learning with Trust Region Methodscited by 18
- 2018Intervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in Navigationcited by 12
- 2021Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Servicecited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.