# Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Setting

arXiv: https://arxiv.org/abs/2202.02580 · HTML: https://mlsanity.com/paper/2202.02580/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 5 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning](https://mlsanity.com/paper/1905.06731/) (2019)
- [Global Convergence of ADMM in Nonconvex Nonsmooth Optimization](https://mlsanity.com/paper/1511.06324/) (2015)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [On the Linear Convergence of the ADMM in Decentralized Consensus Optimization](https://mlsanity.com/paper/1307.5561/) (2013)
- [Optimal Algorithms for Non-Smooth Distributed Optimization in Networks](https://mlsanity.com/paper/1806.00291/) (2018)
- [COLA: Decentralized Linear Learning](https://mlsanity.com/paper/1808.04883/) (2018)
- [Convergence Rate of Distributed ADMM over Networks](https://mlsanity.com/paper/1601.00194/) (2016)
- [Parallel Multi-Block ADMM with o(1/k) Convergence](https://mlsanity.com/paper/1312.3040/) (2013)
- [Ordering for Communication-Efficient Quickest Change Detection in a Decomposable Graphical Model](https://mlsanity.com/paper/2008.03871/) (2020)

## Built on by (most cited first, 4 of 4)

- [Personalized Graph Federated Learning with Differential Privacy](https://mlsanity.com/paper/2306.06399/) (2023)
- [Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentum](https://mlsanity.com/paper/2311.18578/) (2023)
- [Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descent](https://mlsanity.com/paper/2209.11944/) (2022)
- [AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation](https://mlsanity.com/paper/2312.16261/) (2023)

## Abstract (arXiv)

The challenge of communication-efficient distributed optimization has attracted attention in recent years. In this paper, a communication efficient algorithm, called ordering-based alternating direction method of multipliers (OADMM) is devised in a general fully decentralized network setting where a worker can only exchange messages with neighbors. Compared to the classical ADMM, a key feature of OADMM is that transmissions are ordered among workers at each iteration such that a worker with the most informative data broadcasts its local variable to neighbors first, and neighbors who have not transmitted yet can update their local variables based on that received transmission. In OADMM, we prohibit workers from transmitting if their current local variables are not sufficiently different from their previously transmitted value. A variant of OADMM, called SOADMM, is proposed where transmissions are ordered but transmissions are never stopped for each node at each iteration. Numerical results demonstrate that given a targeted accuracy, OADMM can significantly reduce the number of communications compared to existing algorithms including ADMM. We also show numerically that SOADMM can accelerate convergence, resulting in communication savings compared to the classical ADMM.