mlsanityData to 14 Oct 2026

Age-of-information minimization under energy harvesting and non-stationary environment

arXiv 2504.04916 ↗First posted 7 Apr 2025eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

This work focuses on minimizing the age of information for multiple energy harvesting sources that sample data and transmit it to a sink node. At each time, the central scheduler selects one of the sources to probe the quality of its channel to the sink node, and then the assessed channel quality is utilized to determine whether a source will sample and send the packet. For a single source case, we assume that the probed channel quality is known at each time instant, model the problem of AoI minimization as a Markov decision process, and prove the optimal sampling policy threshold structure. We then use this threshold structure and propose an AEC-SW-UCRL2 algorithm to handle unknown and time varying energy harvesting rate and channel statistics, motivated by the popular SWUCRL2 algorithm for non stationary reinforcement learning. This algorithm is applicable when an upper bound is available for the total variation of each of these quantities over a time horizon. Furthermore, in situations where these variation budgets are not accessible, we introduce the AEC-BORL algorithm, motivated by the well known BORL algorithm. For the multiple source case, we demonstrate that the AoI minimization problem can be formulated as a constrained MDP, which can be relaxed using a Lagrange multiplier and decoupled into sub problems across source nodes. We also derive Whittle index based source scheduling policy for probing and an optimal threshold policy for source sampling. We next leverage this Whittle index and threshold structure to develop the WIT-SW-UCRL2 algorithm for unknown time varying energy harvesting rates and channel statistics under their respective variation budgets. Moreover, we also proposed a Whittle index and threshold based bandit over reinforcement learning (WIT-BORL) algorithm for unknown variation budgets. Finally, we numerically demonstrate the efficacy of our algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 56 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

23 of 56 references.

References · 23

  1. 2017A Tutorial on Thompson Samplingcited by 433
  2. 2020Whittle index based Q-learning for restless bandits with average rewardcited by 45
  3. 2020Dynamic Regret of Policy Optimization in Non-stationary Environmentscited by 33
  4. 2018A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitionscited by 30
  5. 2018Age of Information: Whittle Index for Scheduling Stochastic Arrivalscited by 21
  6. 2019A Whittle Index Approach to Minimizing Functions of Age of Informationcited by 21
  7. 2021Q-Learning Lagrange Policies for Multi-Action Restless Banditscited by 21
  8. 2022Optimistic Whittle Index Policy: Online Learning for Restless Banditscited by 20
  9. 2020Model-Free Non-Stationary RL: Near-Optimal Regret and Applications in Multi-Agent RL and Inventory Controlcited by 17
  10. 2018Joint Status Sampling and Updating for Minimizing Age of Information in the Internet of Thingscited by 10
  11. 2019A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systemscited by 10
  12. 2020AoI Minimization in Status Update Control with Energy Harvesting Sensorscited by 10
11 more
  1. 2022On learning Whittle index policy for restless bandits with scalable regretcited by 10
  2. 2021A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQcited by 9
  3. 2021An Online Learning Approach to Optimizing Time-Varying Costs of AoIcited by 6
  4. 2019Closed-Form Whittle's Index-Enabled Random Access for Timely Status Updatecited by 3
  5. 2022On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks with Energy Harvesting Sensorscited by 3
  6. 2022Minimizing the AoI in Resource-Constrained Multi-Source Relaying Systems: Dynamic and Learning-based Schedulingcited by 3
  7. 2021On the Global Optimality of Whittle's index policy for minimizing the age of informationcited by 2
  8. 2022Age-of-information minimization via opportunistic sampling by an energy harvesting sourcecited by 2
  9. 2021Learning to Minimize Age of Information over an Unreliable Channel with Energy Harvestingcited by 1
  10. 2021Whittle Index Based Scheduling Policy for Minimizing the Cost of Age of Informationcited by 1
  11. 2024Whittle's index-based age-of-information minimization in multi-energy harvesting source networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels ↗
  2. 2025An MDP Model for Censoring in Harvesting Sensors: Optimal and Approximated Solutions
  3. 2025Multi-Task Lifelong Reinforcement Learning for Wireless Sensor Networks
  4. 2024Federated Learning With Energy Harvesting Devices: An MDP Framework
  5. 2025Policy Gradient Algorithms for Age-of-Information Cost Minimization
  6. 2025Performance Optimization of Energy-Harvesting Underlay Cognitive Radio Networks Using Reinforcement Learning
  7. 2025Multi-agent Robust and Optimal Policy Learning for Data Harvesting ↗
  8. 2026Joint Freshness and Age-Dispersion Control over Finite-State Markov Wireless Channels ↗