mlsanityData to 7 Oct 2026

Local Environment Poisoning Attacks on Federated Reinforcement Learning

arXiv 2303.02725 ↗First posted 5 Mar 2023cs.LG cs.CRcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) has become a popular tool for solving traditional Reinforcement Learning (RL) tasks. The multi-agent structure addresses the major concern of data-hungry in traditional RL, while the federated mechanism protects the data privacy of individual agents. However, the federated mechanism also exposes the system to poisoning by malicious agents that can mislead the trained policy. Despite the advantage brought by FL, the vulnerability of Federated Reinforcement Learning (FRL) has not been well-studied before. In this work, we propose a general framework to characterize FRL poisoning as an optimization problem and design a poisoning protocol that can be applied to policy-based FRL. Our framework can also be extended to FRL with actor-critic as a local RL algorithm by training a pair of private and public critics. We provably show that our method can strictly hurt the global objective. We verify our poisoning effectiveness by conducting extensive experiments targeting mainstream RL algorithms and over various RL OpenAI Gym environments covering a wide range of difficulty levels. Within these experiments, we compare clean and baseline poisoning methods against our proposed framework. The results show that the proposed framework is successful in poisoning FRL systems and reducing performance across various environments and does so more effectively than baseline methods. Our work provides new insights into the vulnerability of FL in RL training and poses new challenges for designing robust FRL algorithms

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 5 Mar 2023.

Builds on

20 of 36 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  4. 2018How To Backdoor Federated Learningcited by 745
  5. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  6. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  7. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  8. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  9. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  10. 2020Byzantine-Resilient Secure Federated Learningcited by 93
  11. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  12. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
8 more
  1. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 77
  2. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 52
  3. 2019Federated Transfer Reinforcement Learning for Autonomous Drivingcited by 43
  4. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  5. 2019TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agentscited by 29
  6. 2022Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learningcited by 27
  7. 2021Multi-Task Federated Reinforcement Learning with Adversariescited by 13
  8. 2018Security in Mobile Edge Caching with Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 3; 2026: 1.

6 papers.

  1. 2023A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspectivecited by 9
  2. 2025Provably Robust Federated Reinforcement Learningcited by 8
  3. 2025Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defensescited by 2
  4. 2025Gradient Inversion in Federated Reinforcement Learningcited by 1
  5. 2024FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuningcited by 0
  6. 2026When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learningcited by 0