# GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning

arXiv: https://arxiv.org/abs/1909.00047 · HTML: https://mlsanity.com/paper/1909.00047/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 30 of 30)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems](https://mlsanity.com/paper/2101.03367/) (2021)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates](https://mlsanity.com/paper/2310.19807/) (2023)
- [Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space](https://mlsanity.com/paper/2104.00352/) (2021)
- [An Energy and Carbon Footprint Analysis of Distributed and Federated Learning](https://mlsanity.com/paper/2206.10380/) (2022)
- [Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus](https://mlsanity.com/paper/2101.03300/) (2021)
- [Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning](https://mlsanity.com/paper/2206.02604/) (2022)
- [Federated Offline Reinforcement Learning](https://mlsanity.com/paper/2206.05581/) (2022)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [Differentially Private Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2106.06127/) (2021)
- [Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM](https://mlsanity.com/paper/2207.10226/) (2022)
- [A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning](https://mlsanity.com/paper/2103.10346/) (2021)
- [Towards Energy Efficient Distributed Federated Learning for 6G Networks](https://mlsanity.com/paper/2201.08270/) (2022)
- [A Systematic Literature Review on Blockchain Enabled Federated Learning Framework for Internet of Vehicles](https://mlsanity.com/paper/2203.05192/) (2022)
- [Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning](https://mlsanity.com/paper/2203.02865/) (2022)
- [Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs](https://mlsanity.com/paper/2110.01686/) (2021)
- [Distributed Event-Based Learning via ADMM](https://mlsanity.com/paper/2405.10618/) (2024)
- [PGT-I: Scaling Spatiotemporal GNNs with Memory-Efficient Distributed Training](https://mlsanity.com/paper/2507.11683/) (2025)
- [Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices](https://mlsanity.com/paper/2403.09141/) (2024)
- [Matching Pursuit Based Scheduling for Over-the-Air Federated Learning](https://mlsanity.com/paper/2206.06679/) (2022)
- [Edge AI Collaborative Learning: Bayesian Approaches to Uncertainty Estimation](https://mlsanity.com/paper/2410.08651/) (2024)
- [Decentralized Optimization with Topology-Independent Communication](https://mlsanity.com/paper/2509.14488/) (2025)
- [Generalized ADMM in Distributed Learning via Variational Inequality](https://mlsanity.com/paper/2104.12608/) (2021)
- [Differentially Private Federated Learning via Inexact ADMM with Multiple Local Updates](https://mlsanity.com/paper/2202.09409/) (2022)
- [DeFTA: A Plug-and-Play Decentralized Replacement for FedAvg](https://mlsanity.com/paper/2204.02632/) (2022)
- [Decentralized federated learning methods for reducing communication cost and energy consumption in UAV networks](https://mlsanity.com/paper/2304.06551/) (2023)
- [Distributed Linear Regression with Compositional Covariates](https://mlsanity.com/paper/2310.13969/) (2023)
- [Convergence Visualizer of Decentralized Federated Distillation with Reduced Communication Costs](https://mlsanity.com/paper/2312.11905/) (2023)

## Abstract (arXiv)

When the data is distributed across multiple servers, lowering the communication cost between the servers (or workers) while solving the distributed learning problem is an important problem and is the focus of this paper. In particular, we propose a fast, and communication-efficient decentralized framework to solve the distributed machine learning (DML) problem. The proposed algorithm, Group Alternating Direction Method of Multipliers (GADMM) is based on the Alternating Direction Method of Multipliers (ADMM) framework. The key novelty in GADMM is that it solves the problem in a decentralized topology where at most half of the workers are competing for the limited communication resources at any given time. Moreover, each worker exchanges the locally trained model only with two neighboring workers, thereby training a global model with a lower amount of communication overhead in each exchange. We prove that GADMM converges to the optimal solution for convex loss functions, and numerically show that it converges faster and more communication-efficient than the state-of-the-art communication-efficient algorithms such as the Lazily Aggregated Gradient (LAG) and dual averaging, in linear and logistic regression tasks on synthetic and real datasets. Furthermore, we propose Dynamic GADMM (D-GADMM), a variant of GADMM, and prove its convergence under the time-varying network topology of the workers.