# Distributed Reinforcement Learning for Age of Information Minimization in Real-Time IoT Systems

arXiv: https://arxiv.org/abs/2104.01527 · HTML: https://mlsanity.com/paper/2104.01527/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [A Theoretical Analysis of Deep Q-Learning](https://mlsanity.com/paper/1901.00137/) (2019)
- [Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks](https://mlsanity.com/paper/2012.03158/) (2020)
- [Cooperative Internet of UAVs: Distributed Trajectory Design by Multi-agent Deep Reinforcement Learning](https://mlsanity.com/paper/2007.14297/) (2020)
- [Joint Status Sampling and Updating for Minimizing Age of Information in the Internet of Things](https://mlsanity.com/paper/1807.04356/) (2018)
- [A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems](https://mlsanity.com/paper/1908.06367/) (2019)
- [Deep Learning for Optimal Deployment of UAVs with Visible Light Communications](https://mlsanity.com/paper/1912.00752/) (2019)
- [A Machine Learning Approach for Task and Resource Allocation in Mobile Edge Computing Based Networks](https://mlsanity.com/paper/2007.10102/) (2020)
- [Reinforcement Learning Based Scheduling Algorithm for Optimizing Age of Information in Ultra Reliable Low Latency Networks](https://mlsanity.com/paper/1811.06776/) (2018)
- [Optimizing Information Freshness in Computing enabled IoT Networks](https://mlsanity.com/paper/1910.05578/) (2019)
- [Closed-Form Analysis of Non-Linear Age-of-Information in Status Updates with an Energy Harvesting Transmitter](https://mlsanity.com/paper/1906.00192/) (2019)
- [Optimizing Information Freshness for Cooperative IoT Systems with Stochastic Arrivals](https://mlsanity.com/paper/2101.00382/) (2021)

## Built on by (most cited first, 10 of 10)

- [Performance Optimization for Semantic Communications: An Attention-based Reinforcement Learning Approach](https://mlsanity.com/paper/2208.08239/) (2022)
- [Optimizing Communication and Device Clustering for Clustered Federated Learning with Differential Privacy](https://mlsanity.com/paper/2507.07320/) (2025)
- [Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2312.10418/) (2023)
- [Optimizing Wireless Resource Management and Synchronization in Digital Twin Networks](https://mlsanity.com/paper/2502.05116/) (2025)
- [Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions](https://mlsanity.com/paper/2211.01036/) (2022)
- [Deep Reinforcement Learning Based Power Allocation for Minimizing AoI and Energy Consumption in MIMO-NOMA IoT Systems](https://mlsanity.com/paper/2303.06411/) (2023)
- [Decentralized Online Learning in Task Assignment Games for Mobile Crowdsensing](https://mlsanity.com/paper/2309.10594/) (2023)
- [Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning](https://mlsanity.com/paper/2209.08947/) (2022)
- [Collaborative Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Trajectory Design for 3D UAV Tracking](https://mlsanity.com/paper/2401.12079/) (2024)
- [A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning](https://mlsanity.com/paper/2512.21412/) (2025)

## Abstract (arXiv)

In this paper, the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices is studied. In the considered model, each IoT device monitors a physical process that follows nonlinear dynamics. As the dynamics of the physical process vary over time, each device must find an optimal sampling frequency to sample the real-time dynamics of the physical system and send sampled information to a base station (BS). Due to limited wireless resources, the BS can only select a subset of devices to transmit their sampled information. Thus, edge devices must cooperatively sample their monitored dynamics based on the local observations and the BS must collect the sampled information from the devices immediately, hence avoiding the additional time and energy used for sampling and information transmission. To this end, it is necessary to jointly optimize the sampling policy of each device and the device selection scheme of the BS so as to accurately monitor the dynamics of the physical process using minimum energy. This problem is formulated as an optimization problem whose goal is to minimize the weighted sum of AoI cost and energy consumption. To solve this problem, we propose a novel distributed reinforcement learning (RL) approach for the sampling policy optimization. The proposed algorithm enables edge devices to cooperatively find the global optimal sampling policy using their own local observations. Given the sampling policy, the device selection scheme can be optimized thus minimizing the weighted sum of AoI and energy consumption of all devices. Simulations with real data of PM 2.5 pollution show that the proposed algorithm can reduce the sum of AoI by up to 17.8% and 33.9% and the total energy consumption by up to 13.2% and 35.1%, compared to a conventional deep Q network method and a uniform sampling policy.