mlsanityData to 14 Oct 2026

A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability

arXiv 2506.04291 ↗First posted 4 Jun 2025cs.LGcited by 2

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

With the proliferation of Internet of Things (IoT) devices, the demand for addressing complex optimization challenges has intensified. The Lyapunov Drift-Plus-Penalty algorithm is a widely adopted approach for ensuring queue stability, and some research has preliminarily explored its integration with reinforcement learning (RL). In this paper, we investigate the adaptation of the Lyapunov Drift-Plus-Penalty algorithm for RL applications, deriving an effective method for combining Lyapunov Drift-Plus-Penalty with RL under a set of common and reasonable conditions through rigorous theoretical analysis. Unlike existing approaches that directly merge the two frameworks, our proposed algorithm, termed Lyapunov drift-plus-penalty method tailored for reinforcement learning with queue stability (LDPTRLQ) algorithm, offers theoretical superiority by effectively balancing the greedy optimization of Lyapunov Drift-Plus-Penalty with the long-term perspective of RL. Simulation results for multiple problems demonstrate that LDPTRLQ outperforms the baseline methods using the Lyapunov drift-plus-penalty method and RL, corroborating the validity of our theoretical derivations. The results also demonstrate that our proposed algorithm outperforms other benchmarks in terms of compatibility and stability.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 31 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

9 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Achieving Instance-dependent Sample Complexity for Constrained Markov Decision Process26 Feb

Other references · 8

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2020Exploration-Exploitation in Constrained MDPscited by 149
  3. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  4. 2020Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networkscited by 15
  5. 2020Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networkscited by 12
  6. 2024Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approachcited by 6
  7. 2015On the Convergence Time of Dual Subgradient Methods for Strongly Convex Programscited by 1
  8. 2023Energy-Latency Aware Intelligent Reflecting Surface Aided Multi-cell Mobile Edge Computingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Theoretical Foundations of Continual Learning via Drift-Plus-Penaltycited by 1
  2. 2026Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Asynchronous Stochastic Approximation with Applications to Average-Reward Reinforcement Learning
  2. 2026Reinforcement Learning for Control with Probabilistic Stability Guarantee: A Finite-Sample Approach
  3. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency
  4. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  5. 2026Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning
  6. 2026Drift Q-Learning
  7. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  8. 2024Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report