mlsanityData to 14 Oct 2026

Reinforcement Learning with Reward Machines for Sleep Control in Mobile Networks

arXiv 2604.07411 ↗First posted 8 Apr 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Energy efficiency in mobile networks is crucial for sustainable telecommunications infrastructure, particularly as network densification continues to increase power consumption. Sleep mechanisms for the components in mobile networks can reduce energy use, but deciding which components to put to sleep, when, and for how long while preserving quality of service (QoS) remains a difficult optimisation problem. In this paper, we utilise reinforcement learning with reward machines (RMs) to make sleep-control decisions that balance immediate energy savings and long-term QoS impact, i.e. time-averaged packet drop rates for deadline-constrained traffic and time-averaged minimum-throughput guarantees for constant-rate users. A challenge is that time-averaged constraints depend on cumulative performance over time rather than immediate performance. As a result, the effective reward is non-Markovian, and optimal actions depend on operational history rather than the instantaneous system state. RMs account for the history dependence by maintaining an abstract state that explicitly tracks the QoS constraint violations over time. Our framework provides a principled, scalable approach to energy management for next-generation mobile networks under diverse traffic patterns and QoS requirements.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 19 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

6 of 19 references.

References · 6

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2017Constrained Policy Optimizationcited by 983
  3. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  4. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  5. 2021A Survey on 5G Radio Access Network Energy Efficiency: Massive MIMO, Lean Carrier Design, Sleep Modes, and Machine Learningcited by 15
  6. 2024Semantic-Aware Remote Estimation of Multiple Markov Sources Under Constraintscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Average Reward Reinforcement Learning for Wireless Radio Resource Management
  2. 2026Scalable machine learning-based approaches for energy saving in densely deployed Open RAN ↗
  3. 2022A Reinforcement Learning Framework for PQoS in a Teleoperated Driving Scenario
  4. 2024Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems
  5. 2026Mobile Network Control with a World Model
  6. 2026Energy Saving in 5G and Beyond Networks: A Quantum Reinforcement Learning Approach
  7. 2026Temporally Consistent Graph Q-Networks for Intelligent Network Control
  8. 2024Reinforcement Learning for Sustainable Energy: A Survey