# Zeno++: Robust Fully Asynchronous SGD

arXiv: https://arxiv.org/abs/1903.07020 · HTML: https://mlsanity.com/paper/1903.07020/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2019.

## Built on by (most cited first, 30 of 53)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums](https://mlsanity.com/paper/2205.12173/) (2022)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Byzantine-Robust Decentralized Learning via ClippedGossip](https://mlsanity.com/paper/2202.01545/) (2022)
- [AFLGuard: Byzantine-robust Asynchronous Federated Learning](https://mlsanity.com/paper/2212.06325/) (2022)
- [A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy](https://mlsanity.com/paper/2302.10637/) (2023)
- [Zero-Knowledge Proof-based Verifiable Decentralized Machine Learning in Communication Network: A Comprehensive Survey](https://mlsanity.com/paper/2310.14848/) (2023)
- [Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam](https://mlsanity.com/paper/2202.06009/) (2022)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [Secure Byzantine-Robust Distributed Learning via Clustering](https://mlsanity.com/paper/2110.02940/) (2021)
- [Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data](https://mlsanity.com/paper/2203.01214/) (2022)
- [Meta Stackelberg Game: Robust Federated Learning against Adaptive and Mixed Poisoning Attacks](https://mlsanity.com/paper/2410.17431/) (2024)
- [High Dimensional Distributed Gradient Descent with Arbitrary Number of Byzantine Attackers](https://mlsanity.com/paper/2307.13352/) (2023)
- [BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers](https://mlsanity.com/paper/2110.09660/) (2021)
- [Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting](https://mlsanity.com/paper/2302.05049/) (2023)
- [MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attacks](https://mlsanity.com/paper/2207.07941/) (2022)
- [MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering](https://mlsanity.com/paper/2208.10161/) (2022)
- [Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments](https://mlsanity.com/paper/2110.11006/) (2021)
- [ByzSecAgg: A Byzantine-Resistant Secure Aggregation Scheme for Federated Learning Based on Coded Computing and Vector Commitment](https://mlsanity.com/paper/2302.09913/) (2023)
- [Can Membership Inferencing be Refuted?](https://mlsanity.com/paper/2303.03648/) (2023)
- [FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning](https://mlsanity.com/paper/2303.05206/) (2023)
- [Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness](https://mlsanity.com/paper/2404.17970/) (2024)
- [Tolerating Adversarial Attacks and Byzantine Faults in Distributed Machine Learning](https://mlsanity.com/paper/2109.02018/) (2021)
- [Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization](https://mlsanity.com/paper/2211.05554/) (2022)
- [Online Learning with Adversaries: A Differential-Inclusion Analysis](https://mlsanity.com/paper/2304.01525/) (2023)
- [SureFED: Robust Federated Learning via Uncertainty-Aware Inward and Outward Inspection](https://mlsanity.com/paper/2308.02747/) (2023)
- [Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation](https://mlsanity.com/paper/2509.02970/) (2025)
- [Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating](https://mlsanity.com/paper/2303.16241/) (2023)
- [Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML](https://mlsanity.com/paper/2501.09621/) (2025)
- [Adversarial Robustness in Distributed Quantum Machine Learning](https://mlsanity.com/paper/2508.11848/) (2025)

## Abstract (arXiv)

We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers. In contrast to previous work, Zeno++ removes some unrealistic restrictions on worker-server communications, allowing for fully asynchronous updates from anonymous workers, arbitrarily stale worker updates, and the possibility of an unbounded number of Byzantine workers. The key idea is to estimate the descent of the loss value after the candidate gradient is applied, where large descent values indicate that the update results in optimization progress. We prove the convergence of Zeno++ for non-convex problems under Byzantine failures. Experimental results show that Zeno++ outperforms existing approaches.