# Robust Online Learning over Networks

arXiv: https://arxiv.org/abs/2309.00520 · HTML: https://mlsanity.com/paper/2309.00520/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 1 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [On the Convergence of Decentralized Gradient Descent](https://mlsanity.com/paper/1310.7063/) (2013)
- [Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications](https://mlsanity.com/paper/2008.02608/) (2020)
- [Federated Learning: A Signal Processing Perspective](https://mlsanity.com/paper/2103.17150/) (2021)
- [Time-Varying Convex Optimization: Time-Structured Algorithms and Applications](https://mlsanity.com/paper/2006.08500/) (2020)
- [A general framework for decentralized optimization with first-order methods](https://mlsanity.com/paper/2009.05837/) (2020)
- [Linear Convergent Decentralized Optimization with Compression](https://mlsanity.com/paper/2007.00232/) (2020)
- [ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates](https://mlsanity.com/paper/1506.02396/) (2015)
- [Optimization and Learning with Information Streams: Time-varying Algorithms and Applications](https://mlsanity.com/paper/1910.08123/) (2019)
- [Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization](https://mlsanity.com/paper/2009.04373/) (2020)
- [On the O(1/k) Convergence of Asynchronous Distributed Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1307.8254/) (2013)
- [Achieving Linear Convergence in Distributed Asynchronous Multi-agent Optimization](https://mlsanity.com/paper/1803.10359/) (2018)
- [Can Primal Methods Outperform Primal-dual Methods in Decentralized Dynamic Optimization?](https://mlsanity.com/paper/2003.00816/) (2020)
- [Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis](https://mlsanity.com/paper/1509.02597/) (2015)
- [GTAdam: Gradient Tracking with Adaptive Momentum for Distributed Online Optimization](https://mlsanity.com/paper/2009.01745/) (2020)
- [Distributed Optimization for Smart Cyber-Physical Networks](https://mlsanity.com/paper/1906.10760/) (2019)
- [Distributed and Inexact Proximal Gradient Method for Online Convex Optimization](https://mlsanity.com/paper/2001.00870/) (2020)
- [Coordinate Friendly Structures, Algorithms and Applications](https://mlsanity.com/paper/1601.00863/) (2016)
- [SuperMann: a superlinearly convergent algorithm for finding fixed points of nonexpansive operators](https://mlsanity.com/paper/1609.06955/) (2016)
- [tvopt: A Python Framework for Time-Varying Optimization](https://mlsanity.com/paper/2011.07119/) (2020)
- [Dynamic Max-Consensus and Size Estimation of Anonymous Multi-Agent Networks](https://mlsanity.com/paper/2009.03858/) (2020)
- [Distributed Variable Sample-size Stochastic Optimization with Fixed Step-sizes](https://mlsanity.com/paper/2108.05078/) (2021)
- [Stability, Linear Convergence, and Robustness of the Wang-Elia Algorithm for Distributed Consensus Optimization](https://mlsanity.com/paper/2204.10030/) (2022)
- [SI-ADMM: A Stochastic Inexact ADMM Framework for Stochastic Convex Programs](https://mlsanity.com/paper/1711.05286/) (2017)
- [Distributed estimation and control of node centrality in undirected asymmetric networks](https://mlsanity.com/paper/1903.09689/) (2019)
- [A Stochastic Operator Framework for Optimization and Learning with Sub-Weibull Errors](https://mlsanity.com/paper/2105.09884/) (2021)
- [Analysis of Distributed ADMM Algorithm for Consensus Optimization in Presence of Node Error](https://mlsanity.com/paper/1901.02436/) (2019)
- [A Nonlinear Perron-Frobenius Approach for Stability and Consensus of Discrete-Time Multi-Agent Systems](https://mlsanity.com/paper/1907.10461/) (2019)
- [Novel Stability Conditions for Nonlinear Monotone Systems and Consensus in Multi-Agent Networks](https://mlsanity.com/paper/2301.05140/) (2023)

## Built on by (most cited first, 2 of 2)

- [Communication-Efficient Stochastic Distributed Learning](https://mlsanity.com/paper/2501.13516/) (2025)
- [Optimization and Learning in Open Multi-Agent Systems](https://mlsanity.com/paper/2501.16847/) (2025)

## Abstract (arXiv)

The recent deployment of multi-agent networks has enabled the distributed solution of learning problems, where agents cooperate to train a global model without sharing their local, private data. This work specifically targets some prevalent challenges inherent to distributed learning: (i) online training, i.e., the local data change over time; (ii) asynchronous agent computations; (iii) unreliable and limited communications; and (iv) inexact local computations. To tackle these challenges, we apply the Distributed Operator Theoretical (DOT) version of the Alternating Direction Method of Multipliers (ADMM), which we call "DOT-ADMM". We prove that if the DOT-ADMM operator is metric subregular, then it converges with a linear rate for a large class of (not necessarily strongly) convex learning problems toward a bounded neighborhood of the optimal time-varying solution, and characterize how such neighborhood depends on (i)-(iv). We first derive an easy-to-verify condition for ensuring the metric subregularity of an operator, followed by tutorial examples on linear and logistic regression problems. We corroborate the theoretical analysis with numerical simulations comparing DOT-ADMM with other state-of-the-art algorithms, showing that only the proposed algorithm exhibits robustness to (i)-(iv).