# Data Encoding for Byzantine-Resilient Distributed Optimization

arXiv: https://arxiv.org/abs/1907.02664 · HTML: https://mlsanity.com/paper/1907.02664/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 10 of 10)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums](https://mlsanity.com/paper/2205.12173/) (2022)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [LEGATO: A LayerwisE Gradient AggregaTiOn Algorithm for Mitigating Byzantine Attacks in Federated Learning](https://mlsanity.com/paper/2107.12490/) (2021)
- [Online Learning with Adversaries: A Differential-Inclusion Analysis](https://mlsanity.com/paper/2304.01525/) (2023)
- [Coded Robust Aggregation for Distributed Learning under Byzantine Attacks](https://mlsanity.com/paper/2506.01989/) (2025)
- [Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding](https://mlsanity.com/paper/2603.28780/) (2026)
- [Tight Convergence Rates for Online Distributed Linear Estimation with Adversarial Measurements](https://mlsanity.com/paper/2604.06282/) (2026)
- [Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates](https://mlsanity.com/paper/2605.09337/) (2026)
- [Enhanced Byzantine-Robust Federated Learning Via Truncated-Quadratic Loss for Heterogeneous Data](https://mlsanity.com/paper/2607.10970/) (2026)

## Abstract (arXiv)

We study distributed optimization in the presence of Byzantine adversaries, where both data and computation are distributed among m worker machines, t of which may be corrupt. The compromised nodes may collaboratively and arbitrarily deviate from their pre-specified programs, and a designated (master) node iteratively computes the model/parameter vector for generalized linear models. In this work, we primarily focus on two iterative algorithms: Proximal Gradient Descent (PGD) and Coordinate Descent (CD). Gradient descent (GD) is a special case of these algorithms. PGD is typically used in the data-parallel setting, where data is partitioned across different samples, whereas, CD is used in the model-parallelism setting, where data is partitioned across the parameter space. In this paper, we propose a method based on data encoding and error correction over real numbers to combat adversarial attacks. We can tolerate up to t≤ lfloorfracm-12rfloor corrupt worker nodes, which is information-theoretically optimal. We give deterministic guarantees, and our method does not assume any probability distribution on the data. We develop a em sparse encoding scheme which enables computationally efficient data encoding and decoding. We demonstrate a trade-off between the corruption threshold and the resource requirements (storage, computational, and communication complexity). As an example, for t≤fracm3, our scheme incurs only a em constant overhead on these resources, over that required by the plain distributed PGD/CD algorithms which provide no adversarial protection. To the best of our knowledge, ours is the first paper that makes CD secure against adversarial attacks. Our encoding scheme extends efficiently to the data streaming model and for stochastic gradient descent (SGD). We also give experimental results to show the efficacy of our proposed schemes.