# LEGATO: A LayerwisE Gradient AggregaTiOn Algorithm for Mitigating Byzantine Attacks in Federated Learning

arXiv: https://arxiv.org/abs/2107.12490 · HTML: https://mlsanity.com/paper/2107.12490/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [The Hidden Vulnerability of Distributed Learning in Byzantium](https://mlsanity.com/paper/1802.07927/) (2018)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [Federated Optimization:Distributed Optimization Beyond the Datacenter](https://mlsanity.com/paper/1511.03575/) (2015)
- [Mitigating Sybils in Federated Learning Poisoning](https://mlsanity.com/paper/1808.04866/) (2018)
- [Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation](https://mlsanity.com/paper/1903.03936/) (2019)
- [Generalized Byzantine-tolerant SGD](https://mlsanity.com/paper/1802.10116/) (2018)
- [Are All Layers Created Equal?](https://mlsanity.com/paper/1902.01996/) (2019)
- [Learning from Untrusted Data](https://mlsanity.com/paper/1611.02315/) (2016)
- [Robust Federated Learning in a Heterogeneous Environment](https://mlsanity.com/paper/1906.06629/) (2019)
- [DRACO: Byzantine-resilient Distributed Training via Redundant Gradients](https://mlsanity.com/paper/1803.09877/) (2018)
- [Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging](https://mlsanity.com/paper/1909.05125/) (2019)
- [Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy](https://mlsanity.com/paper/1806.00939/) (2018)
- [Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent](https://mlsanity.com/paper/1705.05491/) (2017)
- [IBM Federated Learning: an Enterprise Framework White Paper V0.1](https://mlsanity.com/paper/2007.10987/) (2020)
- [DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation](https://mlsanity.com/paper/1907.12205/) (2019)
- [Attack-Resistant Federated Learning with Residual-based Reweighting](https://mlsanity.com/paper/1912.11464/) (2019)
- [Constant Approximation for k-Median and k-Means with Outliers via Iterative Rounding](https://mlsanity.com/paper/1711.01323/) (2017)
- [Byzantine-Tolerant Machine Learning](https://mlsanity.com/paper/1703.02757/) (2017)
- [Data Encoding for Byzantine-Resilient Distributed Optimization](https://mlsanity.com/paper/1907.02664/) (2019)
- [Distributed Momentum for Byzantine-resilient Learning](https://mlsanity.com/paper/2003.00010/) (2020)
- [Asynchronous Byzantine Machine Learning (the case of SGD)](https://mlsanity.com/paper/1802.07928/) (2018)
- [Client Adaptation improves Federated Learning with Simulated Non-IID Clients](https://mlsanity.com/paper/2007.04806/) (2020)

## Built on by (most cited first, 4 of 4)

- [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023)
- [Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation](https://mlsanity.com/paper/2409.01435/) (2024)
- [Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning](https://mlsanity.com/paper/2208.09894/) (2022)
- [SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Masks](https://mlsanity.com/paper/2312.12484/) (2023)

## Abstract (arXiv)

Federated learning has arisen as a mechanism to allow multiple participants to collaboratively train a model without sharing their data. In these settings, participants (workers) may not trust each other fully; for instance, a set of competitors may collaboratively train a machine learning model to detect fraud. The workers provide local gradients that a central server uses to update a global model. This global model can be corrupted when Byzantine workers send malicious gradients, which necessitates robust methods for aggregating gradients that mitigate the adverse effects of Byzantine inputs. Existing robust aggregation algorithms are often computationally expensive and only effective under strict assumptions. In this paper, we introduce LayerwisE Gradient AggregatTiOn (LEGATO), an aggregation algorithm that is, by contrast, scalable and generalizable. Informed by a study of layer-specific responses of gradients to Byzantine attacks, LEGATO employs a dynamic gradient reweighing scheme that is novel in its treatment of gradients based on layer-specific robustness. We show that LEGATO is more computationally efficient than multiple state-of-the-art techniques and more generally robust across a variety of attack settings in practice. We also demonstrate LEGATO's benefits for gradient descent convergence in the absence of an attack.