# Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2112.10630 · HTML: https://mlsanity.com/paper/2112.10630/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 20 Dec 2021.

## Builds on (most cited first, 15 of 15)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [When Waiting is not an Option : Learning Options with a Deliberation Cost](https://mlsanity.com/paper/1709.04571/) (2017)
- [Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges](https://mlsanity.com/paper/1602.03602/) (2016)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks](https://mlsanity.com/paper/1705.02723/) (2017)
- [Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA](https://mlsanity.com/paper/2002.00073/) (2020)
- [Multicell MIMO Communications Relying on Intelligent Reflecting Surface](https://mlsanity.com/paper/1907.10864/) (2019)
- [An Amateur Drone Surveillance System Based on Cognitive Internet of Things](https://mlsanity.com/paper/1711.10738/) (2017)
- [Robust Secure UAV Communications with the Aid of Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2008.09404/) (2020)
- [Optimization of Wireless Relaying With Flexible UAV-Borne Reflecting Surfaces](https://mlsanity.com/paper/2006.10969/) (2020)
- [On the Trust and Trust Modelling for the Future Fully-Connected Digital World: A Comprehensive Study](https://mlsanity.com/paper/2106.07528/) (2021)
- [Joint Uplink-and-Downlink Optimization of 3D UAV Swarm Deployment for Wireless-Powered NB-IoT Networks](https://mlsanity.com/paper/2008.02993/) (2020)

## Built on by (most cited first, 1 of 1)

- [Graph Attention-based Reinforcement Learning for Trajectory Design and Resource Assignment in Multi-UAV Assisted Communication](https://mlsanity.com/paper/2401.17880/) (2024)

## Abstract (arXiv)

This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network, in which we propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV and also leverage the MUAV as a recharging power source. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network. Depending on whether there is charging between the two UAVs, two optimization problems are formulated. To solve them, two multi-agent deep reinforcement learning (DRL) approaches are proposed, which are centralized training multi-agent deep deterministic policy gradient (CT-MADDPG) and multi-agent deep deterministic policy option critic (MADDPOC). It is shown that the CT-MADDPG can greatly reduce the requirement on the computing capability of the UAV hardware, and the proposed MADDPOC is able to support low-level multi-agent cooperative learning in the continuous action domains, which has great advantages over the existing option-based hierarchical DRL that only support single-agent learning and discrete actions.