mlsanityData to 14 Oct 2026

Provably Robust Federated Reinforcement Learning

arXiv 2502.08123 ↗First posted 12 Feb 2025cs.CR cs.DC cs.LGcited by 8

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Federated reinforcement learning (FRL) allows agents to jointly learn a global decision-making policy under the guidance of a central server. While FRL has advantages, its decentralized design makes it prone to poisoning attacks. To mitigate this, Byzantine-robust aggregation techniques tailored for FRL have been introduced. Yet, in our work, we reveal that these current Byzantine-robust techniques are not immune to our newly introduced Normalized attack. Distinct from previous attacks that targeted enlarging the distance of policy updates before and after an attack, our Normalized attack emphasizes on maximizing the angle of deviation between these updates. To counter these threats, we develop an ensemble FRL approach that is provably secure against both known and our newly proposed attacks. Our ensemble method involves training multiple global policies, where each is learnt by a group of agents using any foundational aggregation rule. These well-trained global policies then individually predict the action for a specific test state. The ultimate action is chosen based on a majority vote for discrete action systems or the geometric median for continuous ones. Our experimental results across different settings show that the Normalized attack can greatly disrupt non-ensemble Byzantine-robust methods, and our ensemble approach offers substantial resistance against poisoning attacks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 54 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

28 of 54 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  3. 2018How To Backdoor Federated Learningcited by 765
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 493
  6. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  7. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 346
  8. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  9. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 194
  10. 2022FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clientscited by 125
  11. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  12. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 82
16 more
  1. 2022DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspectioncited by 81
  2. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 78
  3. 2021Provably Secure Federated Learning against Malicious Clientscited by 63
  4. 2022Federated Reinforcement Learning with Environment Heterogeneitycited by 63
  5. 2017Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descentcited by 59
  6. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 54
  7. 2024Byzantine-Robust Decentralized Federated Learningcited by 48
  8. 2019DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregationcited by 44
  9. 2016Geometric Median in Nearly Linear Timecited by 39
  10. 2022AFLGuard: Byzantine-robust Asynchronous Federated Learningcited by 29
  11. 2016Less than a Single Pass: Stochastically Controlled Stochastic Gradient Methodcited by 18
  12. 2024Poisoning Federated Recommender Systems with Fake Userscited by 15
  13. 2025Do We Really Need to Design New Byzantine-robust Aggregation Rules?cited by 15
  14. 2024Poisoning Attacks on Federated Learning-based Wireless Traffic Predictioncited by 10
  15. 2023Local Environment Poisoning Attacks on Federated Reinforcement Learningcited by 6
  16. 2024On the Hardness of Decentralized Multi-Agent Policy Evaluation under Byzantine Attackscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Toward Malicious Clients Detection in Federated Learningcited by 4
  2. 2026FORLER: Federated Offline Reinforcement Learning with Q-Ensemble and Actor Rectificationcited by 2
  3. 2025Find a Scapegoat: Poisoning Membership Inference Attack and Defense to Federated Learningcited by 1
  4. 2025ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent Systemscited by 0
  5. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0
  6. 2026When the Server Steps In: Calibrated Updates for Fair Federated Learningcited by 0
  7. 2026Online Security Learning in Cooperative Multi-Agent Systems under Hidden Byzantine Attackscited by 0
  8. 2026Fully Byzantine-Resilient Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence
  2. 2024Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
  3. 2026When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning
  4. 2026Learning from Unreliable Trajectories: Adversarially-Robust Federated Q-Learning
  5. 2024FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning
  6. 2025Federated Reinforcement Learning in Heterogeneous Environments
  7. 2021FedCom: A Byzantine-Robust Local Model Aggregation Rule Using Data Commitment for Federated Learning
  8. 2021Federated Ensemble Model-based Reinforcement Learning in Edge Computing