# AoI Minimization in Status Update Control with Energy Harvesting Sensors

arXiv: https://arxiv.org/abs/2009.04224 · HTML: https://mlsanity.com/paper/2009.04224/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [A Learning-Based Trajectory Planning of Multiple UAVs for AoI Minimization in IoT Networks](https://mlsanity.com/paper/2209.09206/) (2022)
- [Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models](https://mlsanity.com/paper/2401.13827/) (2024)
- [Deep Reinforcement Learning Based Power Allocation for Minimizing AoI and Energy Consumption in MIMO-NOMA IoT Systems](https://mlsanity.com/paper/2303.06411/) (2023)
- [Integrated Push-and-Pull Update Model for Goal-Oriented Effective Communication](https://mlsanity.com/paper/2407.14092/) (2024)
- [How to Minimize the Weighted Sum AoI in Multi-Source Status Update Systems: OMA or NOMA?](https://mlsanity.com/paper/2205.03143/) (2022)
- [Age-of-information minimization under energy harvesting and non-stationary environment](https://mlsanity.com/paper/2504.04916/) (2025)
- [AoI-Aware Task Offloading and Transmission Optimization for Industrial IoT Networks: A Branching Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2510.16414/) (2025)
- [Policy Gradient Algorithms for Age-of-Information Cost Minimization](https://mlsanity.com/paper/2512.11990/) (2025)
- [A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning](https://mlsanity.com/paper/2512.21412/) (2025)
- [Adaptive Requesting in Decentralized Edge Networks via Non-Stationary Bandits](https://mlsanity.com/paper/2601.08760/) (2026)

## Abstract (arXiv)

Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control systems. We consider an IoT status update system with multiple users, multiple energy harvesting sensors, and a wireless edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users send requests to the edge node where a cache contains the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send a status update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the age of information of the served measurements. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning methods. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors' battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are analytically characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.