# Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees

arXiv: https://arxiv.org/abs/2002.10940 · HTML: https://mlsanity.com/paper/2002.10940/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 30 of 35)

- [Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise](https://mlsanity.com/paper/2506.04192/) (2025)
- [DoCoFL: Downlink Compression for Cross-Device Federated Learning](https://mlsanity.com/paper/2302.00543/) (2023)
- [Practical Vertical Federated Learning with Unsupervised Representation Learning](https://mlsanity.com/paper/2208.10278/) (2022)
- [Privacy-Aware Compression for Federated Data Analysis](https://mlsanity.com/paper/2203.08134/) (2022)
- [z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learning](https://mlsanity.com/paper/2302.02589/) (2023)
- [Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam](https://mlsanity.com/paper/2202.06009/) (2022)
- [Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation](https://mlsanity.com/paper/2205.00107/) (2022)
- [Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction](https://mlsanity.com/paper/2406.00489/) (2024)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [ScionFL: Efficient and Robust Secure Quantized Aggregation](https://mlsanity.com/paper/2210.07376/) (2022)
- [A Secure Federated Learning Framework for Residential Short Term Load Forecasting](https://mlsanity.com/paper/2209.14547/) (2022)
- [Federated Learning via Plurality Vote](https://mlsanity.com/paper/2110.02998/) (2021)
- [Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design](https://mlsanity.com/paper/2211.03942/) (2022)
- [Byzantine-Resilient Federated Learning at Edge](https://mlsanity.com/paper/2303.10434/) (2023)
- [Privacy Preserving Machine Learning Model Personalization through Federated Personalized Learning](https://mlsanity.com/paper/2505.01788/) (2025)
- [Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz](https://mlsanity.com/paper/2310.14616/) (2023)
- [Masked Random Noise for Communication Efficient Federated Learning](https://mlsanity.com/paper/2408.03220/) (2024)
- [Leveraging Channel Noise for Sampling and Privacy via Quantized Federated Langevin Monte Carlo](https://mlsanity.com/paper/2202.13932/) (2022)
- [Breaking the Communication-Privacy-Accuracy Tradeoff with f-Differential Privacy](https://mlsanity.com/paper/2302.09624/) (2023)
- [Online Learning with Adversaries: A Differential-Inclusion Analysis](https://mlsanity.com/paper/2304.01525/) (2023)
- [Resilience in Online Federated Learning: Mitigating Model-Poisoning Attacks via Partial Sharing](https://mlsanity.com/paper/2403.13108/) (2024)
- [Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity](https://mlsanity.com/paper/2302.09634/) (2023)
- [Binary Federated Learning with Client-Level Differential Privacy](https://mlsanity.com/paper/2308.03320/) (2023)
- [Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization](https://mlsanity.com/paper/2310.15976/) (2023)
- [TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data](https://mlsanity.com/paper/2402.10816/) (2024)
- [Privacy Amplification for the Gaussian Mechanism via Bounded Support](https://mlsanity.com/paper/2403.05598/) (2024)
- [On the Byzantine Fault Tolerance of signSGD with Majority Vote](https://mlsanity.com/paper/2502.19170/) (2025)
- [SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding](https://mlsanity.com/paper/2402.01340/) (2024)
- [Hi-SAFE: Hierarchical Secure Aggregation for Lightweight Federated Learning](https://mlsanity.com/paper/2511.18887/) (2025)
- [Detection and Mitigation of Byzantine Attacks in Distributed Training](https://mlsanity.com/paper/2208.08085/) (2022)

## Abstract (arXiv)

Federated learning (FL) has emerged as a prominent distributed learning paradigm. FL entails some pressing needs for developing novel parameter estimation approaches with theoretical guarantees of convergence, which are also communication efficient, differentially private and Byzantine resilient in the heterogeneous data distribution settings. Quantization-based SGD solvers have been widely adopted in FL and the recently proposed SIGNSGD with majority vote shows a promising direction. However, no existing methods enjoy all the aforementioned properties. In this paper, we propose an intuitively-simple yet theoretically-sound method based on SIGNSGD to bridge the gap. We present Stochastic-Sign SGD which utilizes novel stochastic-sign based gradient compressors enabling the aforementioned properties in a unified framework. We also present an error-feedback variant of the proposed Stochastic-Sign SGD which further improves the learning performance in FL. We test the proposed method with extensive experiments using deep neural networks on the MNIST dataset and the CIFAR-10 dataset. The experimental results corroborate the effectiveness of the proposed method.