# Decentralized Learning with Lazy and Approximate Dual Gradients

arXiv: https://arxiv.org/abs/2008.01748 · HTML: https://mlsanity.com/paper/2008.01748/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 6 of 6)

- [Exponential Graph is Provably Efficient for Decentralized Deep Training](https://mlsanity.com/paper/2110.13363/) (2021)
- [Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2204.10607/) (2022)
- [Communication-Efficient ADMM-based Federated Learning](https://mlsanity.com/paper/2110.15318/) (2021)
- [FedGiA: An Efficient Hybrid Algorithm for Federated Learning](https://mlsanity.com/paper/2205.01438/) (2022)
- [A dual approach for federated learning](https://mlsanity.com/paper/2201.11183/) (2022)
- [Distributed Event-Based Learning via ADMM](https://mlsanity.com/paper/2405.10618/) (2024)

## Abstract (arXiv)

This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neighbors. A line of recent research in this area focuses on improving both computation and communication complexities. The methods SSDA and MSDA citescaman2017optimal have optimal communication complexity when the objective is smooth and strongly convex, and are simple to derive. However, they require solving a subproblem at each step. We propose new algorithms that save computation through using (stochastic) gradients and saves communications when previous information is sufficiently useful. Our methods remain relatively simple -- rather than solving a subproblem, they run Katyusha for a small, fixed number of steps from the latest point. An easy-to-compute, local rule is used to decide if a worker can skip a round of communication. Furthermore, our methods provably reduce communication and computation complexities of SSDA and MSDA. In numerical experiments, our algorithms achieve significant computation and communication reduction compared with the state-of-the-art.