# Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model

arXiv: https://arxiv.org/abs/2112.00333 · HTML: https://mlsanity.com/paper/2112.00333/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Neural Combinatorial Optimization with Reinforcement Learning](https://mlsanity.com/paper/1611.09940/) (2016)
- [Reinforcement Learning for Solving the Vehicle Routing Problem](https://mlsanity.com/paper/1802.04240/) (2018)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://mlsanity.com/paper/1812.11794/) (2018)
- [Deep Reinforcement Learning for Multi-objective Optimization](https://mlsanity.com/paper/1906.02386/) (2019)
- [Seq2Slate: Re-ranking and Slate Optimization with RNNs](https://mlsanity.com/paper/1810.02019/) (2018)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination](https://mlsanity.com/paper/1907.00123/) (2019)
- [UAV Trajectory Planning in Wireless Sensor Networks for Energy Consumption Minimization by Deep Reinforcement Learning](https://mlsanity.com/paper/2108.00354/) (2021)
- [Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2003.04816/) (2020)
- [Trajectory Design for Energy Minimization in UAV-enabled Wireless Communications with Latency Constraints](https://mlsanity.com/paper/1910.08612/) (2019)
- [Energy Minimization in UAV-Aided Networks: Actor-Critic Learning for Constrained Scheduling Optimization](https://mlsanity.com/paper/2006.13610/) (2020)
- [Optimal Resonant Beam Charging for Electronic Vehicles in Internet of Intelligent Vehicles](https://mlsanity.com/paper/1809.09373/) (2018)

## Abstract (arXiv)

Employing unmanned aerial vehicles (UAVs) has attracted growing interests and emerged as the state-of-the-art technology for data collection in Internet-of-Things (IoT) networks. In this paper, with the objective of minimizing the total energy consumption of the UAV-IoT system, we formulate the problem of jointly designing the UAV's trajectory and selecting cluster heads in the IoT network as a constrained combinatorial optimization problem which is classified as NP-hard and challenging to solve. We propose a novel deep reinforcement learning (DRL) with a sequential model strategy that can effectively learn the policy represented by a sequence-to-sequence neural network for the UAV's trajectory design in an unsupervised manner. Through extensive simulations, the obtained results show that the proposed DRL method can find the UAV's trajectory that requires much less energy consumption when compared to other baseline algorithms and achieves close-to-optimal performance. In addition, simulation results show that the trained model by our proposed DRL algorithm has an excellent generalization ability to larger problem sizes without the need to retrain the model.