mlsanityData to 7 Oct 2026

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

arXiv 2204.03529 ↗First posted 7 Apr 2022cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is an emerging framework for distributed processing of large data volumes by edge devices subject to limited communication bandwidths, heterogeneity in data distributions and computational resources, as well as privacy considerations. In this paper, we introduce a new FL protocol termed FedADMM based on primal-dual optimization. The proposed method leverages dual variables to tackle statistical heterogeneity, and accommodates system heterogeneity by tolerating variable amount of work performed by clients. FedADMM maintains identical communication costs per round as FedAvg/Prox, and generalizes them via the augmented Lagrangian. A convergence proof is established for nonconvex objectives, under no restrictions in terms of data dissimilarity or number of participants per round of the algorithm. We demonstrate the merits through extensive experiments on real datasets, under both IID and non-IID data distributions across clients. FedADMM consistently outperforms all baseline methods in terms of communication efficiency, with the number of rounds needed to reach a prescribed accuracy reduced by up to 87%. The algorithm effectively adapts to heterogeneous data distributions through the use of dual variables, without the need for hyperparameter tuning, and its advantages are more pronounced in large-scale systems.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

21 of 35 references.

References · 21

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2019Towards Federated Learning at Scale: System Designcited by 826
  6. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  7. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  8. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  9. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  10. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  11. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  12. 2018In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learningcited by 80
9 more
  1. 2016CoCoA: A General Framework for Communication-Efficient Distributed Optimizationcited by 69
  2. 2014Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problemscited by 64
  3. 2016Training Neural Networks Without Gradients: A Scalable ADMM Approachcited by 61
  4. 2019Federated Learning with Bayesian Differential Privacycited by 60
  5. 2014Multi-Agent Distributed Optimization via Inexact Consensus ADMMcited by 21
  6. 2015Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysiscited by 10
  7. 2021FedHybrid: A Hybrid Primal-Dual Algorithm Framework for Federated Optimizationcited by 7
  8. 2021DN-ADMM: Distributed Newton ADMM for Multi-agent Optimizationcited by 1
  9. 2021BFGS-ADMM for Large-Scale Distributed Optimizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 7; 2025: 6; 2026: 2.

25 papers.

Sort:

  1. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  2. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  3. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  4. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14
  5. 2022Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Frameworkcited by 9
  6. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  7. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  8. 2023Federated Learning with Convex Global and Local Constraintscited by 4
  9. 2024FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learningcited by 4
  10. 2025Connecting Federated ADMM to Bayescited by 4
  11. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  12. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3
13 more
  1. 2024Distributed Event-Based Learning via ADMMcited by 3
  2. 2025Federated ADMM from Bayesian Dualitycited by 3
  3. 2024On the Convergence of Federated Learning Algorithms without Data Similaritycited by 2
  4. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  5. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  6. 2025Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networkscited by 2
  7. 2024Controlling Participation in Federated Learning with Feedbackcited by 1
  8. 2025FedAPM: Federated Learning via ADMM with Partial Model Personalizationcited by 1
  9. 2023Personalized Federated Learning via ADMM with Moreau Envelopecited by 0
  10. 2024On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairnesscited by 0
  11. 2025Sharpness-aware Federated Graph Learningcited by 0
  12. 2026First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraintscited by 0
  13. 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalizationcited by 0