mlsanityData to 14 Oct 2026

Communication-Efficient Distributionally Robust Decentralized Learning

arXiv 2205.15614 ↗First posted 31 May 2022cs.LG cs.AI eess.SPcited by 10

Not on the map. Why?

Abstract (arXiv)

Decentralized learning algorithms empower interconnected devices to share data and computational resources to collaboratively train a machine learning model without the aid of a central coordinator. In the case of heterogeneous data distributions at the network nodes, collaboration can yield predictors with unsatisfactory performance for a subset of the devices. For this reason, in this work, we consider the formulation of a distributionally robust decentralized learning task and we propose a decentralized single loop gradient descent/ascent algorithm (AD-GDA) to directly solve the underlying minimax optimization problem. We render our algorithm communication-efficient by employing a compressed consensus scheme and we provide convergence guarantees for smooth convex and non-convex loss functions. Finally, we corroborate the theoretical findings with empirical results that highlight AD-GDA's ability to provide unbiased predictors and to greatly improve communication efficiency compared to existing distributionally robust algorithms.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

30 of 59 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  6. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  7. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  8. 2019Agnostic Federated Learningcited by 456
  9. 2017Robust Adversarial Reinforcement Learningcited by 451
  10. 2018Sparsified SGD with Memorycited by 376
  11. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  12. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
18 more
  1. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 291
  2. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  3. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
  4. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  5. 2018The Convergence of Sparsified Gradient Methodscited by 229
  6. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 221
  7. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  8. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 111
  9. 2021Distributionally Robust Federated Averagingcited by 104
  10. 2021Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysiscited by 51
  11. 2019A Decentralized Parallel Algorithm for Training Generative Adversarial Netscited by 43
  12. 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
  13. 2022Federated Minimax Optimization: Improved Convergence Analyses and Algorithmscited by 37
  14. 2020Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Gamescited by 26
  15. 2019A Decentralized Proximal Point-type Method for Saddle Point Problemscited by 17
  16. 2020Decentralized SGD with Over-the-Air Computationcited by 16
  17. 2022Asynchronous Decentralized Learning over Unreliable Wireless Networkscited by 13
  18. 2019The Cells Out of Sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifierscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 1; 2026: 2.

10 papers.

  1. 2023Scaff-PD: Communication Efficient Fair and Robust Federated Learningcited by 6
  2. 2022DR-DSGD: A Distributionally Robust Decentralized Learning Algorithm over Graphscited by 5
  3. 2024Communication-Efficient Federated Group Distributionally Robust Optimizationcited by 3
  4. 2023Balancing Energy Efficiency and Distributional Robustness in Over-the-Air Federated Learningcited by 1
  5. 2024FDR-SVM: A Federated Distributionally Robust Support Vector Machine via a Mixture of Wasserstein Balls Ambiguity Setcited by 1
  6. 2025Energy-Efficient Federated Learning for AIoT using Clustering Methodscited by 1
  7. 2023Switch and Conquer: Efficient Algorithms By Switching Stochastic Gradient Oracles For Decentralized Saddle Point Problemscited by 0
  8. 2024Federated Learning with Relative Fairnesscited by 0
  9. 2026On Gossip Algorithms for Machine Learning with Pairwise Objectivescited by 0
  10. 2026Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compressioncited by 0