# Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2107.11015 · HTML: https://mlsanity.com/paper/2107.11015/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges](https://mlsanity.com/paper/1602.03602/) (2016)
- [UAV Communications for 5G and Beyond: Recent Advances and Future Trends](https://mlsanity.com/paper/1901.06637/) (2019)
- [Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks](https://mlsanity.com/paper/2003.00391/) (2020)
- [Optimal 3D-Trajectory Design and Resource Allocation for Solar-Powered UAV Communication Systems](https://mlsanity.com/paper/1808.00101/) (2018)
- [Path Design for Cellular-Connected UAV with Reinforcement Learning](https://mlsanity.com/paper/1905.03440/) (2019)
- [Joint Trajectory and Resource Allocation Design for Energy-Efficient Secure UAV Communication Systems](https://mlsanity.com/paper/2003.07028/) (2020)
- [Multi-Antenna UAV Data Harvesting: Joint Trajectory and Communication Optimization](https://mlsanity.com/paper/2002.09849/) (2020)

## Built on by (most cited first, 7 of 7)

- [Deep Reinforcement Learning for Trajectory Path Planning and Distributed Inference in Resource-Constrained UAV Swarms](https://mlsanity.com/paper/2212.11201/) (2022)
- [Is That Rain? Understanding Effects on Visual Odometry Performance for Autonomous UAVs and Efficient DNN-based Rain Classification at the Edge](https://mlsanity.com/paper/2407.12663/) (2024)
- [Bridging the Urban-Rural Connectivity Gap through Intelligent Space, Air, and Ground Networks](https://mlsanity.com/paper/2202.12683/) (2022)
- [SHM-Traffic: DRL and Transfer learning based UAV Control for Structural Health Monitoring of Bridges with Traffic](https://mlsanity.com/paper/2402.14757/) (2024)
- [Trajectory Design for UAV-Based Low-Altitude Wireless Networks in Unknown Environments: A Digital Twin-Assisted TD3 Approach](https://mlsanity.com/paper/2510.24255/) (2025)
- [Optimizing Energy and Data Collection in UAV-aided IoT Networks using Attention-based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2601.14092/) (2026)
- [DRL-Enabled Trajectory Planing for UAV-Assisted VLC: Optimal Altitude and Reward Design](https://mlsanity.com/paper/2601.22512/) (2026)

## Abstract (arXiv)

In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a sophisticated three-dimensional (3D) environment, where the UAV's trajectory is optimized to efficiently collect data from multiple IoT ground nodes. Unlike existing approaches focusing only on a simplified two-dimensional scenario and the availability of perfect channel state information (CSI), this paper considers a practical 3D urban environment with imperfect CSI, where the UAV's trajectory is designed to minimize data collection completion time subject to practical throughput and flight movement constraints. Specifically, inspired from the state-of-the-art deep reinforcement learning approaches, we leverage the twin-delayed deep deterministic policy gradient (TD3) to design the UAV's trajectory and present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. In particular, we set an additional information, i.e., the merged pheromone, to represent the state information of UAV and environment as a reference of reward which facilitates the algorithm design. By taking the service statuses of IoT nodes, the UAV's position, and the merged pheromone as input, the proposed algorithm can continuously and adaptively learn how to adjust the UAV's movement strategy. By interacting with the external environment in the corresponding Markov decision process, the proposed algorithm can achieve a near-optimal navigation strategy. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.