# Federated Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1901.08277 · HTML: https://mlsanity.com/paper/1901.08277/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 30 of 35)

- [Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning](https://mlsanity.com/paper/2102.12920/) (2021)
- [Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning](https://mlsanity.com/paper/2401.15273/) (2024)
- [Federated Neural Bandits](https://mlsanity.com/paper/2205.14309/) (2022)
- [Multi-Task Federated Reinforcement Learning with Adversaries](https://mlsanity.com/paper/2103.06473/) (2021)
- [Federated Offline Reinforcement Learning](https://mlsanity.com/paper/2206.05581/) (2022)
- [Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges](https://mlsanity.com/paper/2111.07392/) (2021)
- [Differentially Private Graph Classification with GNNs](https://mlsanity.com/paper/2202.02575/) (2022)
- [Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning](https://mlsanity.com/paper/2311.00201/) (2023)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation](https://mlsanity.com/paper/2305.06446/) (2023)
- [FedFormer: Contextual Federation with Attention in Reinforcement Learning](https://mlsanity.com/paper/2205.13697/) (2022)
- [Federated Reinforcement Learning for Collective Navigation of Robotic Swarms](https://mlsanity.com/paper/2202.01141/) (2022)
- [FedHQL: Federated Heterogeneous Q-Learning](https://mlsanity.com/paper/2301.11135/) (2023)
- [FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning](https://mlsanity.com/paper/2402.05541/) (2024)
- [Federated UCBVI: Communication-Efficient Federated Regret Minimization with Heterogeneous Agents](https://mlsanity.com/paper/2410.22908/) (2024)
- [Network Support for High-performance Distributed Machine Learning](https://mlsanity.com/paper/2102.03394/) (2021)
- [PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications](https://mlsanity.com/paper/2204.12064/) (2022)
- [Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources](https://mlsanity.com/paper/2306.08364/) (2023)
- [Robust Convergence in Federated Learning through Label-wise Clustering](https://mlsanity.com/paper/2112.14244/) (2021)
- [Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence](https://mlsanity.com/paper/2401.03489/) (2024)
- [CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screening](https://mlsanity.com/paper/2403.20156/) (2024)
- [Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis](https://mlsanity.com/paper/2503.17454/) (2025)
- [Convergence Guarantees for Federated SARSA with Local Training and Heterogeneous Agents](https://mlsanity.com/paper/2512.17688/) (2025)
- [Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning](https://mlsanity.com/paper/2212.08973/) (2022)
- [Privacy-Preserving Joint Edge Association and Power Optimization for the Internet of Vehicles via Federated Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.11014/) (2023)
- [Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiency](https://mlsanity.com/paper/2405.17471/) (2024)
- [Federated Offline Policy Optimization with Dual Regularization](https://mlsanity.com/paper/2405.17474/) (2024)
- [FedSlate:A Federated Deep Reinforcement Learning Recommender System](https://mlsanity.com/paper/2409.14872/) (2024)
- [Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments](https://mlsanity.com/paper/2510.07436/) (2025)
- [Decision-Focused Federated Learning Under Heterogeneous Objectives and Constraints](https://mlsanity.com/paper/2604.20031/) (2026)

## Abstract (arXiv)

In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.