mlsanityData to 14 Oct 2026

Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee

arXiv 2110.14074 ↗First posted 26 Oct 2021cs.LG cs.AIcited by 54

Not on the map. Why?

Cited by 22 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The growing literature of Federated Learning (FL) has recently inspired Federated Reinforcement Learning (FRL) to encourage multiple agents to federatively build a better decision-making policy without sharing raw trajectories. Despite its promising applications, existing works on FRL fail to I) provide theoretical analysis on its convergence, and II) account for random system failures and adversarial attacks. Towards this end, we propose the first FRL framework the convergence of which is guaranteed and tolerant to less than half of the participating agents being random system failures or adversarial attackers. We prove that the sample efficiency of the proposed framework is guaranteed to improve with the number of agents and is able to account for such potential failures or attacks. All theoretical results are empirically verified on various RL benchmark tasks.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 39 of its 84 references that are in the map. First posted on arXiv 26 Oct 2021.

Builds on

39 of 84 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  9. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  10. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  11. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  12. 2019Challenges of Real-World Reinforcement Learningcited by 291
27 more
  1. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  2. 2019Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Drivingcited by 152
  3. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 145
  4. 2018Stochastic Variance-Reduced Policy Gradientcited by 106
  5. 2020Differentially-Private Federated Linear Banditscited by 83
  6. 2018Byzantine Stochastic Gradient Descentcited by 81
  7. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  8. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 78
  9. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  10. 2016Variance Reduction for Faster Non-Convex Optimizationcited by 72
  11. 2020Collaborative Machine Learning with Incentive-Aware Model Rewardscited by 71
  12. 2021Federated Multi-armed Bandits with Personalizationcited by 62
  13. 2017Non-convex Finite-Sum Optimization Via SCSG Methodscited by 61
  14. 2019An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradientcited by 59
  15. 2017Stochastic Variance Reduction Methods for Policy Evaluationcited by 58
  16. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 57
  17. 2020Federated Bandit: A Gossiping Approachcited by 55
  18. 2019Practical Federated Gradient Boosting Decision Treescited by 54
  19. 2018Policy Optimization via Importance Samplingcited by 47
  20. 2019Federated Transfer Reinforcement Learning for Autonomous Drivingcited by 44
  21. 2015Learning Contact-Rich Manipulation Skills with Guided Policy Searchcited by 38
  22. 2016Less than a Single Pass: Stochastically Controlled Stochastic Gradient Methodcited by 18
  23. 2017Stochastic Variance Reduction for Policy Gradient Estimationcited by 8
  24. 2019Byzantine Resilient Non-Convex SVRG with Distributed Batch Gradient Computationscited by 6
  25. 2019SVRG for Policy Evaluation with Fewer Gradient Evaluationscited by 5
  26. 2013Gaussian Process-Based Decentralized Data Fusion and Active Sensing for Mobility-on-Demand Systemcited by 4
  27. 2017Gaussian Process Decentralized Data Fusion Meets Transfer Learning in Large-Scale Distributed Cooperative Perceptioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 8; 2023: 11; 2024: 17; 2025: 8; 2026: 8.

54 papers.

Sort:

By area:

  1. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
  2. 2023The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyondcited by 36
  3. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  4. 2024Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learningcited by 31
  5. 2022FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergencecited by 29
  6. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  7. 2022Federated Neural Banditscited by 17
  8. 2022Byzantine-Robust Online and Offline Distributed Reinforcement Learningcited by 13
  9. 2022Federated Offline Reinforcement Learningcited by 13
  10. 2024FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHFcited by 12
  11. 2024Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Sufficescited by 11
  12. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systemscited by 11
38 more
  1. 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
  2. 2022Byzantine-Robust Federated Linear Banditscited by 8
  3. 2022FedFormer: Contextual Federation with Attention in Reinforcement Learningcited by 8
  4. 2023Fair yet Asymptotically Equal Collaborative Learningcited by 8
  5. 2024Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environmentscited by 8
  6. 2025Provably Robust Federated Reinforcement Learningcited by 8
  7. 2023Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learningcited by 7
  8. 2023FedHQL: Federated Heterogeneous Q-Learningcited by 6
  9. 2023Local Environment Poisoning Attacks on Federated Reinforcement Learningcited by 6
  10. 2024Global Convergence Guarantees for Federated Policy Gradient Methods with Adversariescited by 6
  11. 2024Compressed Federated Reinforcement Learning with a Generative Modelcited by 6
  12. 2024Corruption-Robust Offline Two-Player Zero-Sum Markov Gamescited by 4
  13. 2024On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representationscited by 4
  14. 2025Gap-Dependent Bounds for Federated Q-learningcited by 4
  15. 2023Client Selection for Federated Policy Optimization with Environment Heterogeneitycited by 3
  16. 2023Byzantine-Resilient Decentralized Multi-Armed Banditscited by 3
  17. 2024Federated reinforcement learning for robot motion planning with zero-shot generalizationcited by 3
  18. 2024On the Hardness of Decentralized Multi-Agent Policy Evaluation under Byzantine Attackscited by 3
  19. 2024Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergencecited by 2
  20. 2024Federated Prompt-based Decision Transformer for Customized VR Services in Mobile Edge Computing Systemcited by 2
  21. 2024CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screeningcited by 2
  22. 2025Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysiscited by 2
  23. 2025Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Samplingcited by 2
  24. 2025The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learningcited by 2
  25. 2025Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learningcited by 2
  26. 2026Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocolcited by 2
  27. 2022On the Convergence Theory of Meta Reinforcement Learning with Personalized Policiescited by 1
  28. 2023Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithmscited by 1
  29. 2023Harnessing the Power of Federated Learning in Federated Contextual Banditscited by 1
  30. 2024Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiencycited by 1
  31. 2024Federated Offline Policy Optimization with Dual Regularizationcited by 1
  32. 2026FedPref: Federated Preference Learning for Structured Radiology Report Extractioncited by 1
  33. 2023Loss- and Reward-Weighting for Efficient Distributed Reinforcement Learningcited by 0
  34. 2024Single-Loop Federated Actor-Critic across Heterogeneous Environmentscited by 0
  35. 2025Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learningcited by 0
  36. 2025Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learningcited by 0
  37. 2026FedQHD: Closed-Form Function-Space Federated Reinforcement Learningcited by 0
  38. 2026Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learningcited by 0

50 most cited of 54 All 54 (JSON)