# L-FGADMM: Layer-Wise Federated Group ADMM for Communication Efficient Decentralized Deep Learning

arXiv: https://arxiv.org/abs/1911.03654 · HTML: https://mlsanity.com/paper/1911.03654/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2204.10607/) (2022)
- [Communication-Efficient ADMM-based Federated Learning](https://mlsanity.com/paper/2110.15318/) (2021)
- [Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM](https://mlsanity.com/paper/2207.10226/) (2022)
- [FedGiA: An Efficient Hybrid Algorithm for Federated Learning](https://mlsanity.com/paper/2205.01438/) (2022)
- [Mobilizing Personalized Federated Learning in Infrastructure-Less and Heterogeneous Environments via Random Walk Stochastic ADMM](https://mlsanity.com/paper/2304.12534/) (2023)
- [Federated PCA on Grassmann Manifold for IoT Anomaly Detection](https://mlsanity.com/paper/2407.07421/) (2024)
- [Generalized ADMM in Distributed Learning via Variational Inequality](https://mlsanity.com/paper/2104.12608/) (2021)
- [The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset](https://mlsanity.com/paper/2108.03508/) (2021)

## Abstract (arXiv)

This article proposes a communication-efficient decentralized deep learning algorithm, coined layer-wise federated group ADMM (L-FGADMM). To minimize an empirical risk, every worker in L-FGADMM periodically communicates with two neighbors, in which the periods are separately adjusted for different layers of its deep neural network. A constrained optimization problem for this setting is formulated and solved using the stochastic version of GADMM proposed in our prior work. Numerical evaluations show that by less frequently exchanging the largest layer, L-FGADMM can significantly reduce the communication cost, without compromising the convergence speed. Surprisingly, despite less exchanged information and decentralized operations, intermittently skipping the largest layer consensus in L-FGADMM creates a regularizing effect, thereby achieving the test accuracy as high as federated learning (FL), a baseline method with the entire layer consensus by the aid of a central entity.