# Queue-Aware Graph Reinforcement Learning for UAV-ISAC-Assisted Maritime Data Collection

arXiv: https://arxiv.org/abs/2607.00324 · HTML: https://mlsanity.com/paper/2607.00324/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 17 of 17)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Integrated Sensing and Communications: Towards Dual-functional Wireless Networks for 6G and Beyond](https://mlsanity.com/paper/2108.07165/) (2021)
- [Graph Neural Networks for Wireless Communications: From Theory to Practice](https://mlsanity.com/paper/2203.10800/) (2022)
- [Learning Decentralized Wireless Resource Allocations with Graph Neural Networks](https://mlsanity.com/paper/2107.01489/) (2021)
- [A Vision and Framework for the High Altitude Platform Station (HAPS) Networks of the Future](https://mlsanity.com/paper/2007.15088/) (2020)
- [Joint Maneuver and Beamforming Design for UAV-Enabled Integrated Sensing and Communication](https://mlsanity.com/paper/2110.02857/) (2021)
- [Throughput Maximization for UAV-enabled Integrated Periodic Sensing and Communication](https://mlsanity.com/paper/2203.06358/) (2022)
- [Sensing as a Service in 6G Perceptive Networks: A Unified Framework for ISAC Resource Allocation](https://mlsanity.com/paper/2202.09969/) (2022)
- [UAV-enabled Integrated Sensing and Communication: Tracking Design and Optimization](https://mlsanity.com/paper/2401.03726/) (2024)
- [Energy Model for UAV Communications: Experimental Validation and Model Generalization](https://mlsanity.com/paper/2005.01305/) (2020)
- [Energy-Efficient Trajectory Design for UAV-Aided Maritime Data Collection in Wind](https://mlsanity.com/paper/2112.13396/) (2021)
- [A Survey on UAV-Aided Maritime Communications: Deployment Considerations, Applications, and Future Challenges](https://mlsanity.com/paper/2209.09605/) (2022)
- [UAV-Enabled Integrated Sensing and Communication in Maritime Emergency Networks](https://mlsanity.com/paper/2408.14027/) (2024)
- [Mobile Cell-Free Massive MIMO with Multi-Agent Reinforcement Learning: A Scalable Framework](https://mlsanity.com/paper/2412.02581/) (2024)
- [UAV-Enabled Joint Sensing, Communication, Powering and Backhaul Transmission in Maritime Monitoring Networks](https://mlsanity.com/paper/2505.12190/) (2025)

## Abstract (arXiv)

This paper studies high-altitude platform (HAP)-assisted sparse cooperative integrated sensing and communication (ISAC) for UAV-enabled ocean monitoring. A fleet of rotary-wing UAVs senses drifting buoys, collects their monitoring data, and reports local posterior estimates to a HAP that performs fusion and sparse cooperation control. The model explicitly accounts for a spatially correlated sea-patch field, patch-aware buoy dynamics, RCS- and clutter-aware echo sensing, fused posterior Cram'er-Rao bounds (PCRBs), and propulsion-energy-limited UAV mobility. The long-horizon objective is cast as a queue-weighted buffered-collection Markov decision process rather than instantaneous throughput, where each buoy maintains a backlog of buffered observations. The resulting long-horizon design is formulated as a mixed discrete-continuous problem with sensing, communication, mobility, safety, buffered-collection, and onboard-energy constraints. To address the combinatorial association component without replacing learning by a deterministic optimizer, we propose a structured feasible-association graph-MARL framework. A heterogeneous graph encoder produces candidate-edge logits, and a masked sequential b-matching policy samples legal UAV-buoy associations while exactly satisfying UAV-load and buoy-cluster constraints. A MAPPO-style training procedure, an independent queue-state value critic, and a consistency-verification protocol are then specified to support reproducible training. Simulation results on congested maritime scenarios show that the proposed policy improves the cumulative queue-weighted collection utility by about 106% over the rate-driven deterministic decoder, maintains a large margin across sea-state sweeps and medium-to-heavy traffic loads, and transfers to larger networks without fine-tuning.