# FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

arXiv: https://arxiv.org/abs/2204.03529 · HTML: https://mlsanity.com/paper/2204.03529/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [On the Convergence of Local Descent Methods in Federated Learning](https://mlsanity.com/paper/1910.14425/) (2019)
- [FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data](https://mlsanity.com/paper/2005.11418/) (2020)
- [In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning](https://mlsanity.com/paper/1809.07857/) (2018)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [Training Neural Networks Without Gradients: A Scalable ADMM Approach](https://mlsanity.com/paper/1605.02026/) (2016)
- [Federated Learning with Bayesian Differential Privacy](https://mlsanity.com/paper/1911.10071/) (2019)
- [Multi-Agent Distributed Optimization via Inexact Consensus ADMM](https://mlsanity.com/paper/1402.6065/) (2014)
- [Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis](https://mlsanity.com/paper/1509.02597/) (2015)
- [FedHybrid: A Hybrid Primal-Dual Algorithm Framework for Federated Optimization](https://mlsanity.com/paper/2106.01279/) (2021)
- [DN-ADMM: Distributed Newton ADMM for Multi-agent Optimization](https://mlsanity.com/paper/2109.14243/) (2021)
- [BFGS-ADMM for Large-Scale Distributed Optimization](https://mlsanity.com/paper/2109.14804/) (2021)

## Built on by (most cited first, 25 of 25)

- [FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy](https://mlsanity.com/paper/2302.10429/) (2023)
- [Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape](https://mlsanity.com/paper/2305.11584/) (2023)
- [Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization](https://mlsanity.com/paper/2306.05706/) (2023)
- [Fusion of Global and Local Knowledge for Personalized Federated Learning](https://mlsanity.com/paper/2302.11051/) (2023)
- [Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework](https://mlsanity.com/paper/2212.01519/) (2022)
- [DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning](https://mlsanity.com/paper/2308.08290/) (2023)
- [Knowledge Adaptation as Posterior Correction](https://mlsanity.com/paper/2506.14262/) (2025)
- [Federated Learning with Convex Global and Local Constraints](https://mlsanity.com/paper/2310.10117/) (2023)
- [FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learning](https://mlsanity.com/paper/2402.13989/) (2024)
- [Connecting Federated ADMM to Bayes](https://mlsanity.com/paper/2501.17325/) (2025)
- [Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learning](https://mlsanity.com/paper/2310.03461/) (2023)
- [Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentum](https://mlsanity.com/paper/2311.18578/) (2023)
- [Distributed Event-Based Learning via ADMM](https://mlsanity.com/paper/2405.10618/) (2024)
- [Federated ADMM from Bayesian Duality](https://mlsanity.com/paper/2506.13150/) (2025)
- [On the Convergence of Federated Learning Algorithms without Data Similarity](https://mlsanity.com/paper/2403.02347/) (2024)
- [A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs](https://mlsanity.com/paper/2409.18915/) (2024)
- [Learnable Sparse Customization in Heterogeneous Edge Computing](https://mlsanity.com/paper/2412.07216/) (2024)
- [Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks](https://mlsanity.com/paper/2501.11267/) (2025)
- [Controlling Participation in Federated Learning with Feedback](https://mlsanity.com/paper/2411.19242/) (2024)
- [FedAPM: Federated Learning via ADMM with Partial Model Personalization](https://mlsanity.com/paper/2506.04672/) (2025)
- [First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints](https://mlsanity.com/paper/2603.05774/) (2026)
- [Personalized Federated Learning via ADMM with Moreau Envelope](https://mlsanity.com/paper/2311.06756/) (2023)
- [On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness](https://mlsanity.com/paper/2407.16397/) (2024)
- [Sharpness-aware Federated Graph Learning](https://mlsanity.com/paper/2512.16247/) (2025)
- [Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization](https://mlsanity.com/paper/2604.12768/) (2026)

## Abstract (arXiv)

Federated Learning (FL) is an emerging framework for distributed processing of large data volumes by edge devices subject to limited communication bandwidths, heterogeneity in data distributions and computational resources, as well as privacy considerations. In this paper, we introduce a new FL protocol termed FedADMM based on primal-dual optimization. The proposed method leverages dual variables to tackle statistical heterogeneity, and accommodates system heterogeneity by tolerating variable amount of work performed by clients. FedADMM maintains identical communication costs per round as FedAvg/Prox, and generalizes them via the augmented Lagrangian. A convergence proof is established for nonconvex objectives, under no restrictions in terms of data dissimilarity or number of participants per round of the algorithm. We demonstrate the merits through extensive experiments on real datasets, under both IID and non-IID data distributions across clients. FedADMM consistently outperforms all baseline methods in terms of communication efficiency, with the number of rounds needed to reach a prescribed accuracy reduced by up to 87%. The algorithm effectively adapts to heterogeneous data distributions through the use of dual variables, without the need for hyperparameter tuning, and its advantages are more pronounced in large-scale systems.