# Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2404.07450 · HTML: https://mlsanity.com/paper/2404.07450/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 62 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Revolutionizing Future Connectivity: A Contemporary Survey on AI-empowered Satellite-based Non-Terrestrial Networks in 6G](https://mlsanity.com/paper/2303.01633/) (2023)
- [Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing](https://mlsanity.com/paper/2202.12028/) (2022)
- [Multi-Objective Optimization for UAV Swarm-Assisted IoT with Virtual Antenna Arrays](https://mlsanity.com/paper/2308.01511/) (2023)
- [Network Characteristics of LEO Satellite Constellations: A Starlink-Based Measurement from End Users](https://mlsanity.com/paper/2212.13697/) (2022)
- [Handover Protocol Learning for LEO Satellite Networks: Access Delay and Collision Minimization](https://mlsanity.com/paper/2310.20215/) (2023)
- [Enhancement of Direct LEO Satellite-to-Smartphone Communications by Distributed Beamforming](https://mlsanity.com/paper/2308.05055/) (2023)
- [UAV-Enabled Data Collection for Wireless Sensor Networks with Distributed Beamforming](https://mlsanity.com/paper/2004.11332/) (2020)
- [Downlink Coverage and Rate Analysis of Low Earth Orbit Satellite Constellations Using Stochastic Geometry](https://mlsanity.com/paper/2004.13378/) (2020)
- [Rate Splitting Multiple Access for Next Generation Cognitive Radio Enabled LEO Satellite Networks](https://mlsanity.com/paper/2208.03705/) (2022)
- [Distributed Massive MIMO for LEO Satellite Networks](https://mlsanity.com/paper/2211.00832/) (2022)
- [Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2402.04865/) (2024)

## Built on by (most cited first, 8 of 8)

- [Aerial Reliable Collaborative Communications for Terrestrial Mobile Users via Evolutionary Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2502.05824/) (2025)
- [Joint Resource Management for Energy-efficient UAV-assisted SWIPT-MEC: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2505.03230/) (2025)
- [LLM-Aided Joint Secrecy Precoding and Trajectory for RSMA-Based Heterogeneous UAV Networks](https://mlsanity.com/paper/2507.17188/) (2025)
- [Joint Link Adaptation and Device Scheduling Approach for URLLC Industrial IoT Network: A DRL-based Method with Bayesian Optimization](https://mlsanity.com/paper/2512.23493/) (2025)
- [AoI-Sensitive Data Forwarding with Distributed Beamforming in UAV-Assisted IoT](https://mlsanity.com/paper/2502.09038/) (2025)
- [Joint AoI and Handover Optimization in Space-Air-Ground Integrated Network](https://mlsanity.com/paper/2509.12716/) (2025)
- [Enhanced Evolutionary Multi-Objective Deep Reinforcement Learning for Reliable and Efficient Wireless Rechargeable Sensor Networks](https://mlsanity.com/paper/2510.21127/) (2025)
- [Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2605.25916/) (2026)

## Abstract (arXiv)

In this paper, we propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. Specifically, DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas, forming a virtual antenna array to enhance the terminal-to-satellite uplink achievable rates and durations. However, such systems need multiple trade-off policies that variously balance the terminal-satellite uplink achievable rate, energy consumption of terminals, and satellite switching frequency to satisfy the scenario requirement changes. Thus, we perform a multi-objective optimization analysis and formulate a long-term optimization problem. To address availability in different terminal cluster scales, we reformulate this problem into an action space-reduced and universal multi-objective Markov decision process. Then, we propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies, in which the low-value actions are masked to speed up the training process. As such, the applicability of a one-time trained model can cover more changing terminal-satellite uplink scenarios. Simulation results show that the proposed algorithm outmatches various baselines, and draw some useful insights. Specifically, it is found that DCB enables terminals that cannot reach the uplink achievable threshold to achieve efficient direct uplink transmission, which thus reveals that DCB is an effective solution for enabling direct ground-space communications. Moreover, it reveals that the proposed algorithm achieves multiple policies favoring different objectives and achieving near-optimal uplink achievable rates with low switching frequency.