# Federated Deep Reinforcement Learning for the Distributed Control of NextG Wireless Networks

arXiv: https://arxiv.org/abs/2112.03465 · HTML: https://mlsanity.com/paper/2112.03465/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 7 of 7)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Federated Learning for Industrial Internet of Things in Future Industries](https://mlsanity.com/paper/2105.14659/) (2021)
- [Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1712.08266/) (2017)
- [Deep Reinforcement Learning Based Spectrum Allocation in Integrated Access and Backhaul Networks](https://mlsanity.com/paper/2004.13133/) (2020)
- [Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks](https://mlsanity.com/paper/2009.06681/) (2020)
- [Resource allocation in OFDMA networks with half-duplex and imperfect full-duplex users](https://mlsanity.com/paper/1605.01947/) (2016)

## Built on by (most cited first, 6 of 6)

- [FedFormer: Contextual Federation with Attention in Reinforcement Learning](https://mlsanity.com/paper/2205.13697/) (2022)
- [Federated Learning-based MARL for Strengthening Physical-Layer Security in B5G Networks](https://mlsanity.com/paper/2507.06997/) (2025)
- [Matching DNN Compression and Cooperative Training with Resources and Data Availability](https://mlsanity.com/paper/2212.02304/) (2022)
- [FedCritic: Serverless Federated Critic Learning-based Resource Allocation for Multi-Cell OFDMA in 6G](https://mlsanity.com/paper/2605.21418/) (2026)
- [Percentile-Based Deep Reinforcement Learning and Reward Based Personalization For Delay Aware RAN Slicing in O-RAN](https://mlsanity.com/paper/2507.18111/) (2025)
- [FedQoS: Federated QoS-Risk Learning for Heterogeneous Indoor-Outdoor Access Selection](https://mlsanity.com/paper/2608.25496/) (2026)

## Abstract (arXiv)

Next Generation (NextG) networks are expected to support demanding tactile internet applications such as augmented reality and connected autonomous vehicles. Whereas recent innovations bring the promise of larger link capacity, their sensitivity to the environment and erratic performance defy traditional model-based control rationales. Zero-touch data-driven approaches can improve the ability of the network to adapt to the current operating conditions. Tools such as reinforcement learning (RL) algorithms can build optimal control policy solely based on a history of observations. Specifically, deep RL (DRL), which uses a deep neural network (DNN) as a predictor, has been shown to achieve good performance even in complex environments and with high dimensional inputs. However, the training of DRL models require a large amount of data, which may limit its adaptability to ever-evolving statistics of the underlying environment. Moreover, wireless networks are inherently distributed systems, where centralized DRL approaches would require excessive data exchange, while fully distributed approaches may result in slower convergence rates and performance degradation. In this paper, to address these challenges, we propose a federated learning (FL) approach to DRL, which we refer to federated DRL (F-DRL), where base stations (BS) collaboratively train the embedded DNN by only sharing models' weights rather than training data. We evaluate two distinct versions of F-DRL, value and policy based, and show the superior performance they achieve compared to distributed and centralized DRL.