# Adaptive Consensus: A network pruning approach for decentralized optimization

arXiv: https://arxiv.org/abs/2309.02626 · HTML: https://mlsanity.com/paper/2309.02626/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Proving the Lottery Ticket Hypothesis: Pruning is All You Need](https://mlsanity.com/paper/2002.00585/) (2020)
- [EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization](https://mlsanity.com/paper/1404.6264/) (2014)
- [Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphs](https://mlsanity.com/paper/1607.03218/) (2016)
- [NEXT: In-Network Nonconvex Optimization](https://mlsanity.com/paper/1602.00591/) (2016)
- [An Improved Analysis of Gradient Tracking for Decentralized Machine Learning](https://mlsanity.com/paper/2202.03836/) (2022)
- [Push-Pull Gradient Methods for Distributed Optimization in Networks](https://mlsanity.com/paper/1810.06653/) (2018)
- [Communication-Efficient Algorithms for Decentralized and Stochastic Optimization](https://mlsanity.com/paper/1701.03961/) (2017)
- [An Exact Quantized Decentralized Gradient Descent Algorithm](https://mlsanity.com/paper/1806.11536/) (2018)
- [Balancing Communication and Computation in Distributed Optimization](https://mlsanity.com/paper/1709.02999/) (2017)
- [Data-Efficient Decentralized Visual SLAM](https://mlsanity.com/paper/1710.05772/) (2017)
- [Asynchronous Gradient-Push](https://mlsanity.com/paper/1803.08950/) (2018)
- [A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data](https://mlsanity.com/paper/1511.02746/) (2015)
- [Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss](https://mlsanity.com/paper/1501.00263/) (2015)
- [On the Convergence of Nested Decentralized Gradient Methods with Multiple Consensus and Gradient Steps](https://mlsanity.com/paper/2006.01665/) (2020)
- [Balancing Communication and Computation in Gradient Tracking Algorithms for Decentralized Optimization](https://mlsanity.com/paper/2303.14289/) (2023)
- [Nested Distributed Gradient Methods with Adaptive Quantized Communication](https://mlsanity.com/paper/1903.08149/) (2019)
- [Maximal Dissent: a State-Dependent Way to Agree in Distributed Convex Optimization](https://mlsanity.com/paper/2205.00647/) (2022)
- [A Stochastic Gradient Tracking Algorithm for Decentralized Optimization With Inexact Communication](https://mlsanity.com/paper/2307.14942/) (2023)

## Abstract (arXiv)

We consider network-based decentralized optimization problems, where each node in the network possesses a local function and the objective is to collectively attain a consensus solution that minimizes the sum of all the local functions. A major challenge in decentralized optimization is the reliance on communication which remains a considerable bottleneck in many applications. To address this challenge, we propose an adaptive randomized communication-efficient algorithmic framework that reduces the volume of communication by periodically tracking the disagreement error and judiciously selecting the most influential and effective edges at each node for communication. Within this framework, we present two algorithms: Adaptive Consensus (AC) to solve the consensus problem and Adaptive Consensus based Gradient Tracking (AC-GT) to solve smooth strongly convex decentralized optimization problems. We establish strong theoretical convergence guarantees for the proposed algorithms and quantify their performance in terms of various algorithmic parameters under standard assumptions. Finally, numerical experiments showcase the effectiveness of the framework in significantly reducing the information exchange required to achieve a consensus solution.