# Model-aided Federated Reinforcement Learning for Multi-UAV Trajectory Planning in IoT Networks

arXiv: https://arxiv.org/abs/2306.02029 · HTML: https://mlsanity.com/paper/2306.02029/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 3 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Multi-UAV Path Planning for Wireless Data Harvesting with Deep Reinforcement Learning](https://mlsanity.com/paper/2010.12461/) (2020)
- [Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks](https://mlsanity.com/paper/2003.00391/) (2020)
- [Autonomous Recharging and Flight Mission Planning for Battery-operated Autonomous Drones](https://mlsanity.com/paper/1703.10049/) (2017)
- [Model-aided Deep Reinforcement Learning for Sample-efficient UAV Trajectory Design in IoT Networks](https://mlsanity.com/paper/2104.10403/) (2021)

## Built on by (most cited first, 3 of 3)

- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Optimizing Energy and Data Collection in UAV-aided IoT Networks using Attention-based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2601.14092/) (2026)
- [Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment](https://mlsanity.com/paper/2603.16141/) (2026)

## Abstract (arXiv)

Deploying teams of unmanned aerial vehicles (UAVs) to harvest data from distributed Internet of Things (IoT) devices requires efficient trajectory planning and coordination algorithms. Multi-agent reinforcement learning (MARL) has emerged as a solution, but requires extensive and costly real-world training data. To tackle this challenge, we propose a novel model-aided federated MARL algorithm to coordinate multiple UAVs on a data harvesting mission with only limited knowledge about the environment. The proposed algorithm alternates between building an environment simulation model from real-world measurements, specifically learning the radio channel characteristics and estimating unknown IoT device positions, and federated QMIX training in the simulated environment. Each UAV agent trains a local QMIX model in its simulated environment and continuously consolidates it through federated learning with other agents, accelerating the learning process. A performance comparison with standard MARL algorithms demonstrates that our proposed model-aided FedQMIX algorithm reduces the need for real-world training experiences by around three magnitudes while attaining similar data collection performance.