# On-board Deep Q-Network for UAV-assisted Online Power Transfer and Data Collection

arXiv: https://arxiv.org/abs/1906.07064 · HTML: https://mlsanity.com/paper/1906.07064/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 11 of 11)

- [Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey](https://mlsanity.com/paper/2211.04324/) (2022)
- [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023)
- [LLM-Enabled In-Context Learning for Data Collection Scheduling in UAV-assisted Sensor Networks](https://mlsanity.com/paper/2504.14556/) (2025)
- [World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networks](https://mlsanity.com/paper/2506.00417/) (2025)
- [3D UAV Trajectory and Data Collection Optimisation via Deep Reinforcement Learning](https://mlsanity.com/paper/2106.03129/) (2021)
- [Multi-Carrier NOMA-Empowered Wireless Federated Learning with Optimal Power and Bandwidth Allocation](https://mlsanity.com/paper/2302.06730/) (2023)
- [FRSICL: LLM-Enabled In-Context Learning Flight Resource Allocation for Fresh Data Collection in UAV-Assisted Wildfire Monitoring](https://mlsanity.com/paper/2507.10134/) (2025)
- [Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial](https://mlsanity.com/paper/2602.19534/) (2026)
- [Digital Twin-Aided Learning for Managing Reconfigurable Intelligent Surface-Assisted, Uplink, User-Centric Cell-Free Systems](https://mlsanity.com/paper/2302.05073/) (2023)
- [AIC-VDS: Attention-Based In-Context Learning for Joint Velocity Control and Data Collection Scheduling in Multi-UAV-Assisted Pipeline Monitoring](https://mlsanity.com/paper/2510.05698/) (2025)
- [Autonomous Strike UAVs for Counterterrorism Missions: Challenges and Preliminary Solutions](https://mlsanity.com/paper/2403.01022/) (2024)

## Abstract (arXiv)

Unmanned Aerial Vehicles (UAVs) with Microwave Power Transfer (MPT) capability provide a practical means to deploy a large number of wireless powered sensing devices into areas with no access to persistent power supplies. The UAV can charge the sensing devices remotely and harvest their data. A key challenge is online MPT and data collection in the presence of on-board control of a UAV (e.g., patrolling velocity) for preventing battery drainage and data queue overflow of the sensing devices, while up-to-date knowledge on battery level and data queue of the devices is not available at the UAV. In this paper, an on-board deep Q-network is developed to minimize the overall data packet loss of the sensing devices, by optimally deciding the device to be charged and interrogated for data collection, and the instantaneous patrolling velocity of the UAV. Specifically, we formulate a Markov Decision Process (MDP) with the states of battery level and data queue length of sensing devices, channel conditions, and waypoints given the trajectory of the UAV; and solve it optimally with Q-learning. Furthermore, we propose the on-board deep Q-network that can enlarge the state space of the MDP, and a deep reinforcement learning based scheduling algorithm that asymptotically derives the optimal solution online, even when the UAV has only outdated knowledge on the MDP states. Numerical results demonstrate that the proposed deep reinforcement learning algorithm reduces the packet loss by at least 69.2%, as compared to existing non-learning greedy algorithms.