# Fully Byzantine-Resilient Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2609.25701 · HTML: https://mlsanity.com/paper/2609.25701/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)](https://mlsanity.com/paper/2008.00742/) (2020)
- [BRIDGE: Byzantine-resilient Decentralized Gradient Descent](https://mlsanity.com/paper/1908.08098/) (2019)
- [Multi-Agent Reinforcement Learning in Stochastic Networked Systems](https://mlsanity.com/paper/2006.06555/) (2020)
- [Byzantine-Robust Decentralized Learning via ClippedGossip](https://mlsanity.com/paper/2202.01545/) (2022)
- [Byzantine-resilient Decentralized Stochastic Gradient Descent](https://mlsanity.com/paper/2002.08569/) (2020)
- [Byzantine-Resilient Decentralized Stochastic Optimization with Robust Aggregation Rules](https://mlsanity.com/paper/2206.04568/) (2022)
- [Distributed Optimization Under Adversarial Nodes](https://mlsanity.com/paper/1606.08939/) (2016)
- [Adversarial attacks in consensus-based multi-agent reinforcement learning](https://mlsanity.com/paper/2103.06967/) (2021)
- [Robust Multi-Agent LLMs under Byzantine Faults](https://mlsanity.com/paper/2605.09076/) (2026)
- [A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network](https://mlsanity.com/paper/2505.05103/) (2025)
- [Provably Robust Federated Reinforcement Learning](https://mlsanity.com/paper/2502.08123/) (2025)
- [Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees](https://mlsanity.com/paper/2110.05597/) (2021)
- [Distributed Neural Policy Gradient Algorithm for Global Convergence of Networked Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2505.24113/) (2025)
- [Towards Resilience for Multi-Agent QD-Learning](https://mlsanity.com/paper/2104.03153/) (2021)
- [Resilient Distributed Optimization for Multi-Agent Cyberphysical Systems](https://mlsanity.com/paper/2212.02459/) (2022)
- [Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation](https://mlsanity.com/paper/2009.11146/) (2020)
- [On the Hardness of Decentralized Multi-Agent Policy Evaluation under Byzantine Attacks](https://mlsanity.com/paper/2409.12882/) (2024)
- [Distributed Resilience-Aware Control in Multi-Robot Networks](https://mlsanity.com/paper/2504.03120/) (2025)
- [Resilient Average Consensus with Adversaries via Distributed Detection and Recovery](https://mlsanity.com/paper/2405.18752/) (2024)
- [Fully Byzantine-Resilient Distributed Multi-Agent Q-Learning](https://mlsanity.com/paper/2604.02791/) (2026)

## Abstract (arXiv)

We study distributed Byzantine-resilient actor-critic multi-agent reinforcement learning (AC-MARL), where agents collectively learn policies through local interactions. Existing methods guarantee convergence of the agents' parameters only to a neighborhood of the attack-free limit points, resulting in degraded performance. We propose Fully Resilient AC-MARL (FRAC-MARL), a decentralized method in which each agent leverages redundancy in two-hop messages to identify reliable messages. Under linear parameterizations of the value and team-reward functions and Byzantine edge attacks, where adversarial behavior is confined to the communication layer, we prove that agents' parameters converge almost surely to the same limit points as in the attack-free case over time-varying communication graphs. We introduce a novel topological condition for the convergence of our method, present a systematic method to construct such networks, and prove that this condition can be verified in polynomial time. Finally, we demonstrate our method on cooperative multi-robot formation control tasks.