# Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates

arXiv: https://arxiv.org/abs/2605.09337 · HTML: https://mlsanity.com/paper/2605.09337/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 25 of 25)

- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [The Hidden Vulnerability of Distributed Learning in Byzantium](https://mlsanity.com/paper/1802.07927/) (2018)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [Error Feedback Fixes SignSGD and other Gradient Compression Schemes](https://mlsanity.com/paper/1901.09847/) (2019)
- [Lower Bounds for Non-Convex Stochastic Optimization](https://mlsanity.com/paper/1912.02365/) (2019)
- [A Little Is Enough: Circumventing Defenses For Distributed Learning](https://mlsanity.com/paper/1902.06156/) (2019)
- [Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing](https://mlsanity.com/paper/2006.09365/) (2020)
- [Robust Federated Learning in a Heterogeneous Environment](https://mlsanity.com/paper/1906.06629/) (2019)
- [Byzantine Stochastic Gradient Descent](https://mlsanity.com/paper/1803.08917/) (2018)
- [DRACO: Byzantine-resilient Distributed Training via Redundant Gradients](https://mlsanity.com/paper/1803.09877/) (2018)
- [Zeno++: Robust Fully Asynchronous SGD](https://mlsanity.com/paper/1903.07020/) (2019)
- [A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order](https://mlsanity.com/paper/1606.00498/) (2016)
- [AFLGuard: Byzantine-robust Asynchronous Federated Learning](https://mlsanity.com/paper/2212.06325/) (2022)
- [Byzantine-Resilient SGD in High Dimensions on Heterogeneous Data](https://mlsanity.com/paper/2005.07866/) (2020)
- [Data Encoding for Byzantine-Resilient Distributed Optimization](https://mlsanity.com/paper/1907.02664/) (2019)
- [Stochastic Recursive Inclusions in two timescales with non-additive iterate dependent Markov noise](https://mlsanity.com/paper/1611.05961/) (2016)
- [Non-asymptotic bounds for stochastic optimization with biased noisy gradient oracles](https://mlsanity.com/paper/2002.11440/) (2020)
- [Buffered Asynchronous SGD for Byzantine Learning](https://mlsanity.com/paper/2003.00937/) (2020)
- [Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization](https://mlsanity.com/paper/2406.14362/) (2024)
- [Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning](https://mlsanity.com/paper/2502.00193/) (2025)
- [Online Learning with Adversaries: A Differential-Inclusion Analysis](https://mlsanity.com/paper/2304.01525/) (2023)

## Abstract (arXiv)

We propose FAR-SIGN (Fully Asynchronous Robust optimization via SIGNed directional projections) for adversary-resilient learning in parameter-server--worker systems. FAR-SIGN achieves robustness through sign-based updates along carefully designed directions and mitigates the resulting bias via a two-timescale mechanism. It admits both first-order and zeroth-order implementations and enables fully asynchronous execution without requiring a private reference dataset at the server. We establish almost-sure convergence of FAR-SIGN to the set of stationary points for smooth, nonconvex objectives. Moreover, we prove the near-optimal rate of O(n-1/4+ε) in the first-order setting and the standard O(n-1/6+ε) in the zeroth-order setting, where n is the iteration count and ε>0 can be chosen arbitrarily small. Experiments on MNIST show that FAR-SIGN outperforms robust aggregation-based methods in both accuracy and wall-clock time.