# A Practical AoI Scheduler in IoT Networks with Relays

arXiv: https://arxiv.org/abs/2203.04227 · HTML: https://mlsanity.com/paper/2203.04227/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 43 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning](https://mlsanity.com/paper/1809.02121/) (2018)
- [Age of Information: An Introduction and Survey](https://mlsanity.com/paper/2007.08564/) (2020)
- [Hyp-RL : Hyperparameter Optimization by Reinforcement Learning](https://mlsanity.com/paper/1906.11527/) (2019)
- [Cooperative Internet of UAVs: Distributed Trajectory Design by Multi-agent Deep Reinforcement Learning](https://mlsanity.com/paper/2007.14297/) (2020)
- [Deep Reinforcement Learning for Minimizing Age-of-Information in UAV-assisted Networks](https://mlsanity.com/paper/1905.02993/) (2019)
- [Understanding the limits of LoRaWAN](https://mlsanity.com/paper/1607.08011/) (2016)
- [When Machine Learning Meets Wireless Cellular Networks: Deployment, Challenges, and Applications](https://mlsanity.com/paper/1911.03585/) (2019)
- [Experimental Analysis of Safety Application Reliability in V2V Networks](https://mlsanity.com/paper/2005.13031/) (2020)
- [Timely Updates in Energy Harvesting Two-hop Networks: Offline and Online Policies](https://mlsanity.com/paper/1812.01005/) (2018)
- [Unified Analysis of HetNets using Poisson Cluster Process under Max-Power Association](https://mlsanity.com/paper/1812.01830/) (2018)
- [Joint Age of Information and Self Risk Assessment for Safer 802.11p based V2V Networks](https://mlsanity.com/paper/2012.04774/) (2020)
- [AoI-minimizing Scheduling in UAV-relayed IoT Networks](https://mlsanity.com/paper/2107.05181/) (2021)
- [Age of Information in Physical-Layer Network Coding Enabled Two-Way Relay Networks](https://mlsanity.com/paper/2107.06537/) (2021)

## Abstract (arXiv)

Internet of Things (IoT) networks have become ubiquitous as autonomous computing, communication and collaboration among devices become popular for accomplishing various tasks. The use of relays in IoT networks further makes it convenient to deploy IoT networks as relays provide a host of benefits, like increasing the communication range and minimizing power consumption. Existing literature on traditional AoI schedulers for such two-hop relayed IoT networks are limited because they are designed assuming constant/non-changing channel conditions and known (usually, generate-at-will) packet generation patterns. Deep reinforcement learning (DRL) algorithms have been investigated for AoI scheduling in two-hop IoT networks with relays, however, they are only applicable for small-scale IoT networks due to exponential rise in action space as the networks become large. These limitations discourage the practical utilization of AoI schedulers for IoT network deployments. This paper presents a practical AoI scheduler for two-hop IoT networks with relays that addresses the above limitations. The proposed scheduler utilizes a novel voting mechanism based proximal policy optimization (v-PPO) algorithm that maintains a linear action space, enabling it be scale well with larger IoT networks. The proposed v-PPO based AoI scheduler adapts well to changing network conditions and accounts for unknown traffic generation patterns, making it practical for real-world IoT deployments. Simulation results show that the proposed v-PPO based AoI scheduler outperforms both ML and traditional (non-ML) AoI schedulers, such as, Deep Q Network (DQN)-based AoI Scheduler, Maximal Age First-Maximal Age Difference (MAF-MAD), MAF (Maximal Age First) , and round-robin in all considered practical scenarios.