# Decentralized optimization over noisy, rate-constrained networks: Achieving consensus by communicating differences

arXiv: https://arxiv.org/abs/2010.11292 · HTML: https://mlsanity.com/paper/2010.11292/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis](https://mlsanity.com/paper/2101.12704/) (2021)
- [Semi-Decentralized Federated Learning with Collaborative Relaying](https://mlsanity.com/paper/2205.10998/) (2022)
- [Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Framework](https://mlsanity.com/paper/2503.14353/) (2025)
- [Non-Coherent Over-the-Air Decentralized Gradient Descent](https://mlsanity.com/paper/2211.10777/) (2022)
- [DoCoM: Compressed Decentralized Optimization with Near-Optimal Sample Complexity](https://mlsanity.com/paper/2202.00255/) (2022)
- [LASER: Linear Compression in Wireless Distributed Optimization](https://mlsanity.com/paper/2310.13033/) (2023)
- [Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget](https://mlsanity.com/paper/2103.07578/) (2021)
- [Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying](https://mlsanity.com/paper/2202.11850/) (2022)
- [M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion](https://mlsanity.com/paper/2301.09269/) (2023)
- [Convert, compress, correct: Three steps toward communication-efficient DNN training](https://mlsanity.com/paper/2203.09044/) (2022)
- [How to Attain Communication-Efficient DNN Training? Convert, Compress, Correct](https://mlsanity.com/paper/2204.08211/) (2022)
- [Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity](https://mlsanity.com/paper/2607.12172/) (2026)

## Abstract (arXiv)

In decentralized optimization, multiple nodes in a network collaborate to minimize the sum of their local loss functions. The information exchange between nodes required for this task, is often limited by network connectivity. We consider a setting in which communication between nodes is hindered by both (i) a finite rate-constraint on the signal transmitted by any node, and (ii) additive noise corrupting the signal received by any node. We propose a novel algorithm for this scenario: Decentralized Lazy Mirror Descent with Differential Exchanges (DLMD-DiffEx), which guarantees convergence of the local estimates to the optimal solution under the given communication constraints. A salient feature of DLMD-DiffEx is the introduction of additional proxy variables that are maintained by the nodes to account for the disagreement in their estimates due to channel noise and rate-constraints. Convergence to the optimal solution is attained by having nodes iteratively exchange these disagreement terms until consensus is achieved. In order to prevent noise accumulation during this exchange, DLMD-DiffEx relies on two sequences; one controlling the power of the transmitted signal, and the other determining the consensus rate. We provide clear insights on the design of these two sequences which highlights the interplay between consensus rate and noise amplification. We investigate the performance of DLMD-DiffEx both from a theoretical perspective as well as through numerical evaluations.