# Decentralized Federated Reinforcement Learning for User-Centric Dynamic TFDD Control

arXiv: https://arxiv.org/abs/2211.02296 · HTML: https://mlsanity.com/paper/2211.02296/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks](https://mlsanity.com/paper/1810.10408/) (2018)
- [Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation](https://mlsanity.com/paper/2101.06905/) (2021)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Multi-hop RIS-Empowered Terahertz Communications: A DRL-based Hybrid Beamforming Design](https://mlsanity.com/paper/2101.09137/) (2021)
- [Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing](https://mlsanity.com/paper/1906.04594/) (2019)
- [UAV-to-UAV Communications in Cellular Networks](https://mlsanity.com/paper/1912.07534/) (2019)
- [Federated Orchestration for Network Slicing of Bandwidth and Computational Resource](https://mlsanity.com/paper/2002.02451/) (2020)
- [Game-Theoretic Mode Scheduling for Dynamic TDD in 5G Systems](https://mlsanity.com/paper/2104.12367/) (2021)
- [3D Beamforming based Dynamic TDD Interference Mitigation Scheme](https://mlsanity.com/paper/1912.05056/) (2019)

## Built on by (most cited first, 3 of 3)

- [Decision Transformers for RIS-Assisted Systems with Diffusion Model-Based Channel Acquisition](https://mlsanity.com/paper/2501.08007/) (2025)
- [Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach](https://mlsanity.com/paper/2411.14052/) (2024)
- [FedCritic: Serverless Federated Critic Learning-based Resource Allocation for Multi-Cell OFDMA in 6G](https://mlsanity.com/paper/2605.21418/) (2026)

## Abstract (arXiv)

The explosive growth of dynamic and heterogeneous data traffic brings great challenges for 5G and beyond mobile networks. To enhance the network capacity and reliability, we propose a learning-based dynamic time-frequency division duplexing (D-TFDD) scheme that adaptively allocates the uplink and downlink time-frequency resources of base stations (BSs) to meet the asymmetric and heterogeneous traffic demands while alleviating the inter-cell interference. We formulate the problem as a decentralized partially observable Markov decision process (Dec-POMDP) that maximizes the long-term expected sum rate under the users' packet dropping ratio constraints. In order to jointly optimize the global resources in a decentralized manner, we propose a federated reinforcement learning (RL) algorithm named federated Wolpertinger deep deterministic policy gradient (FWDDPG) algorithm. The BSs decide their local time-frequency configurations through RL algorithms and achieve global training via exchanging local RL models with their neighbors under a decentralized federated learning framework. Specifically, to deal with the large-scale discrete action space of each BS, we adopt a DDPG-based algorithm to generate actions in a continuous space, and then utilize Wolpertinger policy to reduce the mapping errors from continuous action space back to discrete action space. Simulation results demonstrate the superiority of our proposed algorithm to benchmark algorithms with respect to system sum rate.