# Byzantine Fault-Tolerance in Peer-to-Peer Distributed Gradient-Descent

arXiv: https://arxiv.org/abs/2101.12316 · HTML: https://mlsanity.com/paper/2101.12316/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2021.

## Built on by (most cited first, 6 of 6)

- [Secure Distributed Training at Scale](https://mlsanity.com/paper/2106.11257/) (2021)
- [Backdoor Attacks in Peer-to-Peer Federated Learning](https://mlsanity.com/paper/2301.09732/) (2023)
- [Byzantine-Resilient Decentralized Multi-Armed Bandits](https://mlsanity.com/paper/2310.07320/) (2023)
- [Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top](https://mlsanity.com/paper/2206.00529/) (2022)
- [Byzantine-Robust Decentralized Stochastic Optimization with Stochastic Gradient Noise-Independent Learning Error](https://mlsanity.com/paper/2308.05292/) (2023)
- [PeerFL: A Simulator for Peer-to-Peer Federated Learning at Scale](https://mlsanity.com/paper/2405.17839/) (2024)

## Abstract (arXiv)

We consider the problem of Byzantine fault-tolerance in the peer-to-peer (P2P) distributed gradient-descent method -- a prominent algorithm for distributed optimization in a P2P system. In this problem, the system comprises of multiple agents, and each agent has a local cost function. In the fault-free case, when all the agents are honest, the P2P distributed gradient-descent method allows all the agents to reach a consensus on a solution that minimizes their aggregate cost. However, we consider a scenario where a certain number of agents may be Byzantine faulty. Such faulty agents may not follow an algorithm correctly, and may share arbitrary incorrect information to prevent other non-faulty agents from solving the optimization problem. In the presence of Byzantine faulty agents, a more reasonable goal is to allow all the non-faulty agents to reach a consensus on a solution that minimizes the aggregate cost of all the non-faulty agents. We refer to this fault-tolerance goal as f-resilience where f is the maximum number of Byzantine faulty agents in a system of n agents, with f < n. Most prior work on fault-tolerance in P2P distributed optimization only consider approximate fault-tolerance wherein, unlike f-resilience, all the non-faulty agents' compute a minimum point of a non-uniformly weighted aggregate of their cost functions. We propose a fault-tolerance mechanism that confers provable f-resilience to the P2P distributed gradient-descent method, provided the non-faulty agents satisfy the necessary condition of 2f-redundancy, defined later in the paper. Moreover, compared to prior work, our algorithm is applicable to a larger class of high-dimensional convex distributed optimization problems.