# QoS-Aware Hierarchical Reinforcement Learning for Joint Link Selection and Trajectory Optimization in SAGIN-Supported UAV Mobility Management

arXiv: https://arxiv.org/abs/2512.15119 · HTML: https://mlsanity.com/paper/2512.15119/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [Accessing From The Sky: A Tutorial on UAV Communications for 5G and Beyond](https://mlsanity.com/paper/1903.05289/) (2019)
- [Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.07574/) (2020)
- [Inter-Plane Inter-Satellite Connectivity in Dense LEO Constellations](https://mlsanity.com/paper/2005.07965/) (2020)
- [Cellular-Enabled UAV Communication: A Connectivity-Constrained Trajectory Optimization Perspective](https://mlsanity.com/paper/1805.07182/) (2018)
- [Learning-based Handover in Mobile Millimeter-wave Networks](https://mlsanity.com/paper/2003.11009/) (2020)
- [ISAC from the Sky: UAV Trajectory Design for Joint Communication and Target Localization](https://mlsanity.com/paper/2207.02904/) (2022)
- [Mobility Management in 5G and Beyond: A Novel Smart Handover with Adaptive Time-to-Trigger and Hysteresis Margin](https://mlsanity.com/paper/2207.01706/) (2022)
- [Non-Terrestrial Networks for UAVs: Base Station Service Provisioning Schemes with Antenna Tilt](https://mlsanity.com/paper/2104.06679/) (2021)
- [Towards Unified AI Models for MU-MIMO Communications: A Tensor Equivariance Framework](https://mlsanity.com/paper/2406.09022/) (2024)
- [Statistical CSI-Based Distributed Precoding Design for OFDM-Cooperative Multi-Satellite Systems](https://mlsanity.com/paper/2505.08038/) (2025)
- [Trajectory Design for Cellular-Connected UAV Under Outage Duration Constraint](https://mlsanity.com/paper/1901.04286/) (2019)
- [A Comprehensive Survey on Moving Networks](https://mlsanity.com/paper/2003.09979/) (2020)
- [Integrated Communications and Localization for Massive MIMO LEO Satellite Systems](https://mlsanity.com/paper/2403.07305/) (2024)
- [Handover-Aware Trajectory Optimization for Cellular-Connected UAV](https://mlsanity.com/paper/2502.19077/) (2025)

## Abstract (arXiv)

Due to the significant variations in unmanned aerial vehicle (UAV) altitude and horizontal mobility, it becomes difficult for any single network to ensure continuous and reliable threedimensional coverage. Towards that end, the space-air-ground integrated network (SAGIN) has emerged as an essential architecture for enabling ubiquitous UAV connectivity. To address the pronounced disparities in coverage and signal characteristics across heterogeneous networks, this paper formulates UAV mobility management in SAGIN as a constrained multi-objective joint optimization problem. The formulation couples discrete link selection with continuous trajectory optimization. Building on this, we propose a two-level multi-agent hierarchical deep reinforcement learning (HDRL) framework that decomposes the problem into two alternately solvable subproblems. To map complex link selection decisions into a compact discrete action space, we conceive a double deep Q-network (DDQN) algorithm in the top-level, which achieves stable and high-quality policy learning through double Q-value estimation. To handle the continuous trajectory action space while satisfying quality of service (QoS) constraints, we integrate the maximum-entropy mechanism of the soft actor-critic (SAC) and employ a Lagrangian-based constrained SAC (CSAC) algorithm in the lower-level that dynamically adjusts the Lagrange multipliers to balance constraint satisfaction and policy optimization. Moreover, the proposed algorithm can be extended to multi-UAV scenarios under the centralized training and decentralized execution (CTDE) paradigm, which enables more generalizable policies. Simulation results demonstrate that the proposed scheme substantially outperforms existing benchmarks in throughput, link switching frequency and QoS satisfaction.