# Modular Distributed Nonconvex Learning with Error Feedback

arXiv: https://arxiv.org/abs/2503.14055 · HTML: https://mlsanity.com/paper/2503.14055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 13 of 13)

- [The Convergence of Sparsified Gradient Methods](https://mlsanity.com/paper/1809.10505/) (2018)
- [Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphs](https://mlsanity.com/paper/1607.03218/) (2016)
- [BEER: Fast O(1/T) Rate for Decentralized Nonconvex Optimization with Communication Compression](https://mlsanity.com/paper/2201.13320/) (2022)
- [Local Exact-Diffusion for Decentralized Optimization and Learning](https://mlsanity.com/paper/2302.00620/) (2023)
- [Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things](https://mlsanity.com/paper/2103.02762/) (2021)
- [Convergence Rate of Distributed ADMM over Networks](https://mlsanity.com/paper/1601.00194/) (2016)
- [Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks](https://mlsanity.com/paper/2106.07243/) (2021)
- [Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation](https://mlsanity.com/paper/2206.03588/) (2022)
- [Error Feedback Reloaded: From Quadratic to Arithmetic Mean of Smoothness Constants](https://mlsanity.com/paper/2402.10774/) (2024)
- [A Tutorial on Distributed Optimization for Cooperative Robotics: from Setups and Algorithms to Toolboxes and Research Directions](https://mlsanity.com/paper/2309.04257/) (2023)
- [ADMM-Tracking Gradient for Distributed Optimization over Asynchronous and Unreliable Networks](https://mlsanity.com/paper/2309.14142/) (2023)
- [A Robust Compressed Push-Pull Method for Decentralized Nonconvex Optimization](https://mlsanity.com/paper/2408.01727/) (2024)
- [Compressed Gradient Tracking Algorithms for Distributed Nonconvex Optimization](https://mlsanity.com/paper/2310.18871/) (2023)

## Abstract (arXiv)

In this paper, we design a novel distributed learning algorithm using stochastic compressed communications. In detail, we pursue a modular approach, merging ADMM and a gradient-based approach, benefiting from the robustness of the former and the computational efficiency of the latter. Additionally, we integrate a stochastic integral action (error feedback) enabling almost sure rejection of the compression error. We analyze the resulting method in nonconvex scenarios and guarantee almost sure asymptotic convergence to the set of stationary points of the problem. This result is obtained using system-theoretic tools based on stochastic timescale separation. We corroborate our findings with numerical simulations in nonconvex classification.