# Federated Deep Reinforcement Learning for THz-Beam Search with Limited CSI

arXiv: https://arxiv.org/abs/2304.13109 · HTML: https://mlsanity.com/paper/2304.13109/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 5 of 5)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design](https://mlsanity.com/paper/2003.00199/) (2020)
- [Hybrid Beamforming for RIS-Empowered Multi-hop Terahertz Communications: A DRL-based Method](https://mlsanity.com/paper/2009.09380/) (2020)
- [Federated Double Deep Q-learning for Joint Delay and Energy Minimization in IoT networks](https://mlsanity.com/paper/2104.11320/) (2021)

## Built on by (most cited first, 2 of 2)

- [AI-Enabled Unmanned Vehicle-Assisted Reconfigurable Intelligent Surfaces: Deployment, Prototyping, Experiments, and Opportunities](https://mlsanity.com/paper/2311.04241/) (2023)
- [Federated Deep Reinforcement Learning for Energy Efficient Multi-Functional RIS-Assisted Low-Earth Orbit Networks](https://mlsanity.com/paper/2501.11079/) (2025)

## Abstract (arXiv)

Terahertz (THz) communication with ultra-wide available spectrum is a promising technique that can achieve the stringent requirement of high data rate in the next-generation wireless networks, yet its severe propagation attenuation significantly hinders its implementation in practice. Finding beam directions for a large-scale antenna array to effectively overcome severe propagation attenuation of THz signals is a pressing need. This paper proposes a novel approach of federated deep reinforcement learning (FDRL) to swiftly perform THz-beam search for multiple base stations (BSs) coordinated by an edge server in a cellular network. All the BSs conduct deep deterministic policy gradient (DDPG)-based DRL to obtain THz beamforming policy with limited channel state information (CSI). They update their DDPG models with hidden information in order to mitigate inter-cell interference. We demonstrate that the cell network can achieve higher throughput as more THz CSI and hidden neurons of DDPG are adopted. We also show that FDRL with partial model update is able to nearly achieve the same performance of FDRL with full model update, which indicates an effective means to reduce communication load between the edge server and the BSs by partial model uploading. Moreover, the proposed FDRL outperforms conventional non-learning-based and existing non-FDRL benchmark optimization methods.