mlsanityData to 14 Oct 2026

Context-aware Constrained Reinforcement Learning Based Energy-Efficient Power Scheduling for Non-stationary XR Data Traffic

arXiv 2503.09391 ↗First posted 12 Mar 2025eess.SY cs.ET cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

In XR downlink transmission, energy-efficient power scheduling (EEPS) is essential for conserving power resource while delivering large data packets within hard-latency constraints. Traditional constrained reinforcement learning (CRL) algorithms show promise in EEPS but still struggle with non-convex stochastic constraints, non-stationary data traffic, and sparse delayed packet dropout feedback (rewards) in XR. To overcome these challenges, this paper models the EEPS in XR as a dynamic parameter-constrained Markov decision process (DP-CMDP) with a varying transition function linked to the non-stationary data traffic and solves it by a proposed context-aware constrained reinforcement learning (CACRL) algorithm, which consists of a context inference (CI) module and a CRL module. The CI module trains an encoder and multiple potential networks to characterize the current transition function and reshape the packet dropout rewards according to the context, transforming the original DP-CMDP into a general CMDP with immediate dense rewards. The CRL module employs a policy network to make EEPS decisions under this CMDP and optimizes the policy using a constrained stochastic successive convex approximation (CSSCA) method, which is better suited for non-convex stochastic constraints. Finally, theoretical analyses provide deep insights into the CADAC algorithm, while extensive simulations demonstrate that it outperforms advanced baselines in both power conservation and satisfying packet dropout constraints.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 65 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

22 of 65 references.

References · 22

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  5. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  6. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  7. 2020Projection-Based Constrained Policy Optimizationcited by 203
  8. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
  9. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 134
  10. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 111
  11. 2019A Finite-Time Analysis of Q-Learning with Neural Network Function Approximationcited by 47
  12. 2021A Primal-Dual Approach to Constrained Markov Decision Processescited by 42
10 more
  1. 2021Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithmcited by 39
  2. 2019Reward Shaping via Meta-Learningcited by 32
  3. 2023Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPscited by 28
  4. 2019Statistical Inference in Mean-Field Variational Bayescited by 10
  5. 2023Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programmingcited by 8
  6. 2023Cancellation-Free Regret Bounds for Lagrangian Approaches in Constrained Markov Decision Processescited by 7
  7. 2021Successive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learningcited by 5
  8. 2021Two-Stage Stochastic Optimization via Primal-Dual Decomposition and Deep Unrollingcited by 4
  9. 2023Generalizable Resource Scaling of 5G Slices using Constrained Reinforcement Learningcited by 4
  10. 2019Enhanced Energy-Efficient Downlink Resource Allocation in Green Non-Orthogonal Multiple Access Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Hybrid Reinforcement Learning Framework for Hard Latency Constrained Resource Scheduling
  2. 2026Reinforcement Learning with Reward Machines for Sleep Control in Mobile Networks
  3. 2025Energy Optimization of Multi-task DNN Inference in MEC-assisted XR Devices: A Lyapunov-Guided Reinforcement Learning Approach
  4. 2025Optimisation of Resource Allocation in Heterogeneous Wireless Networks Using Deep Reinforcement Learning
  5. 2026Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks
  6. 2026A Constrained RL Approach for Cost-Efficient Delivery of Latency-Sensitive Applications
  7. 2026EExApp: GNN-Based Reinforcement Learning for Radio Unit Energy Optimization in 5G O-RAN
  8. 2024Optimizing Wireless Discontinuous Reception via MAC Signaling Learning