PG-Flow: Deterministic implicit policy gradients for geometric product-form queueing networks
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Product-form queueing networks (PFQNs) admit steady-state distributions that factorize into local terms, and in many classical PFQNs including Jackson, BCMP, G-networks, and Energy Packet Networks, these marginals are geometric and parametrized by local flow variables satisfying balance equations. While this structure yields closed-form expressions for key performance metrics, its use for deterministic steady-state optimization remains limited. We introduce PG-Flow, a deterministic policy-gradient framework that differentiates through the steady-state flow fixed-point equations, providing exact gradients via implicit differentiation and a local adjoint system while avoiding trajectory sampling and Poisson equations. We establish global convergence under structural assumptions (affine flow operators and convex local costs), and show that acyclic networks admit linear-time computation of both flows and gradients. Numerical experiments on routing control in Jackson networks and energy-arrival control in Energy Packet Networks demonstrate that PG-Flow provides a principled and computationally efficient approach to deterministic steady-state optimization in geometric product-form networks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 35 references that are in the map. First posted on arXiv 7 Dec 2025.
Builds on
4 of 35 references.
References · 4
- 2021Convex optimizationcited by 2,515
- 2019Deep Declarative Networks: A New Hopecited by 78
- 2024A slot-based energy storage decision-making approach for optimal Off-Grid telecommunication operatorcited by 2
- 2025Efficient Solving of Large Single Input Superstate Decomposable Markovian Decision Processcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Semi-Gradient SARSA Routing with Theoretical Guarantee on Traffic Stability and Weight Convergence
- 2024Differentiable Discrete Event Simulation for Queuing Network Control
- 2024Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems
- 2025Optimal Batched Scheduling of Stochastic Processing Networks Using Atomic Action Decomposition ↗
- 2025Simulation-Driven Reinforcement Learning in Queuing Network Routing Optimization
- 2024Performance of NPG in Countable State-Space Average-Cost RL
- 2024Convergence of Natural Policy Gradient for a Family of Infinite-State Queueing MDPs
- 2025Admission Control of Quasi-Reversible Queueing Systems: Optimization and Reinforcement Learning