# Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models

arXiv: https://arxiv.org/abs/2401.13827 · HTML: https://mlsanity.com/paper/2401.13827/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Towards Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions](https://mlsanity.com/paper/1808.02924/) (2018)
- [Deep Reinforcement Learning for Minimizing Age-of-Information in UAV-assisted Networks](https://mlsanity.com/paper/1905.02993/) (2019)
- [Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks](https://mlsanity.com/paper/2003.00391/) (2020)
- [AoI Minimization in Status Update Control with Energy Harvesting Sensors](https://mlsanity.com/paper/2009.04224/) (2020)
- [Multi-UAV Path Learning for Age and Power Optimization in IoT with UAV Battery Recharge](https://mlsanity.com/paper/2301.03423/) (2023)
- [Traffic Prediction and Fast Uplink for Hidden Markov IoT Models](https://mlsanity.com/paper/2209.05939/) (2022)
- [A Learning-Based Fast Uplink Grant for Massive IoT via Support Vector Machines and Long Short-Term Memory](https://mlsanity.com/paper/2108.10070/) (2021)
- [Machine Learning for Wireless Connectivity and Security of Cellular-Connected UAVs](https://mlsanity.com/paper/1804.05348/) (2018)
- [On the Number and 3D Placement of Drone Base Stations in Wireless Cellular Networks](https://mlsanity.com/paper/1804.08415/) (2018)
- [Multi-tier Drone Architecture for 5G/B5G Cellular Networks: Challenges, Trends, and Prospects](https://mlsanity.com/paper/1711.08407/) (2017)
- [Fast Uplink Grant for NOMA: a Federated Learning based Approach](https://mlsanity.com/paper/1904.07975/) (2019)
- [A Learning-Based Trajectory Planning of Multiple UAVs for AoI Minimization in IoT Networks](https://mlsanity.com/paper/2209.09206/) (2022)

## Built on by (most cited first, 3 of 3)

- [Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.08421/) (2024)
- [An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management](https://mlsanity.com/paper/2501.12991/) (2025)
- [Age and Power Minimization via Meta-Deep Reinforcement Learning in UAV Networks](https://mlsanity.com/paper/2501.14603/) (2025)

## Abstract (arXiv)

The age of information (AoI) is used to measure the freshness of the data. In IoT networks, the traditional resource management schemes rely on a message exchange between the devices and the base station (BS) before communication which causes high AoI, high energy consumption, and low reliability. Unmanned aerial vehicles (UAVs) as flying BSs have many advantages in minimizing the AoI, energy-saving, and throughput improvement. In this paper, we present a novel learning-based framework that estimates the traffic arrival of IoT devices based on Markovian events. The learning proceeds to optimize the trajectory of multiple UAVs and their scheduling policy. First, the BS predicts the future traffic of the devices. We compare two traffic predictors: the forward algorithm (FA) and the long short-term memory (LSTM). Afterward, we propose a deep reinforcement learning (DRL) approach to optimize the optimal policy of each UAV. Finally, we manipulate the optimum reward function for the proposed DRL approach. Simulation results show that the proposed algorithm outperforms the random-walk (RW) baseline model regarding the AoI, scheduling accuracy, and transmission power.