# A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks

arXiv: https://arxiv.org/abs/2508.20645 · HTML: https://mlsanity.com/paper/2508.20645/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 71 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 30 of 38)

- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient](https://mlsanity.com/paper/1703.00102/) (2017)
- [Distributed Stochastic Gradient Tracking Methods](https://mlsanity.com/paper/1805.11454/) (2018)
- [Decentralized Federated Learning: Balancing Communication and Computing Costs](https://mlsanity.com/paper/2107.12048/) (2021)
- [Push-Pull Gradient Methods for Distributed Optimization in Networks](https://mlsanity.com/paper/1810.06653/) (2018)
- [Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergence](https://mlsanity.com/paper/1912.04230/) (2019)
- [Distributed Online Optimization in Dynamic Environments Using Mirror Descent](https://mlsanity.com/paper/1609.02845/) (2016)
- [DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization](https://mlsanity.com/paper/1901.09109/) (2019)
- [Online Optimization with Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit](https://mlsanity.com/paper/1801.07780/) (2018)
- [A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization](https://mlsanity.com/paper/1907.03793/) (2019)
- [Fast Convergence Rates for Distributed Non-Bayesian Learning](https://mlsanity.com/paper/1508.05161/) (2015)
- [A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization](https://mlsanity.com/paper/2102.06752/) (2021)
- [Optimization and Learning with Information Streams: Time-varying Algorithms and Applications](https://mlsanity.com/paper/1910.08123/) (2019)
- [A Distributed Online Convex Optimization Algorithm with Improved Dynamic Regret](https://mlsanity.com/paper/1911.05050/) (2019)
- [An Optimal Hybrid Variance-Reduced Algorithm for Stochastic Composite Nonconvex Optimization](https://mlsanity.com/paper/2008.09055/) (2020)
- [GTAdam: Gradient Tracking with Adaptive Momentum for Distributed Online Optimization](https://mlsanity.com/paper/2009.01745/) (2020)
- [A Compressed Gradient Tracking Method for Decentralized Optimization with Linear Convergence](https://mlsanity.com/paper/2205.12623/) (2022)
- [Distributed Online Optimization for Multi-Agent Networks with Coupled Inequality Constraints](https://mlsanity.com/paper/1805.05573/) (2018)
- [Distributed Detection : Finite-time Analysis and Impact of Network Topology](https://mlsanity.com/paper/1409.8606/) (2014)
- [Coordinate Dual Averaging for Decentralized Online Optimization with Nonseparable Global Objectives](https://mlsanity.com/paper/1508.07933/) (2015)
- [Distributed Online Convex Optimization with an Aggregative Variable](https://mlsanity.com/paper/2007.06844/) (2020)
- [Distributed Personalized Gradient Tracking with Convex Parametric Models](https://mlsanity.com/paper/2008.04363/) (2020)
- [Improving Dynamic Regret in Distributed Online Mirror Descent Using Primal and Dual Information](https://mlsanity.com/paper/2112.03504/) (2021)
- [AB/Push-Pull Method for Distributed Optimization in Time-Varying Directed Networks](https://mlsanity.com/paper/2209.06974/) (2022)
- [Accelerated AB/Push-Pull Methods for Distributed Optimization over Time-Varying Directed Networks](https://mlsanity.com/paper/2302.01214/) (2023)
- [Distributed Optimization over Directed Graphs with Row Stochasticity and Constraint Regularity](https://mlsanity.com/paper/1806.07081/) (2018)
- [An Online Optimization Approach for Multi-Agent Tracking of Dynamic Parameters in the Presence of Adversarial Noise](https://mlsanity.com/paper/1702.06219/) (2017)
- [Distributed Nash Equilibrium Seeking over Time-Varying Directed Communication Networks](https://mlsanity.com/paper/2201.02323/) (2022)

## Abstract (arXiv)

With the increasing scale and dynamics of data, distributed online optimization has become essential for real-time decision-making in various applications. However, existing algorithms often rely on bounded gradient assumptions and overlook the impact of stochastic gradients, especially in time-varying directed networks. This study proposes a novel Time-Varying Hybrid Stochastic Gradient Tracking algorithm named TV-HSGT, based on hybrid stochastic gradient tracking and variance reduction mechanisms. Specifically, TV-HSGT integrates row-stochastic and column-stochastic communication schemes over time-varying digraphs, eliminating the need for Perron vector estimation or out-degree information. By combining current and recursive stochastic gradients, it effectively reduces gradient variance while accurately tracking global descent directions. Theoretical analysis demonstrates that TV-HSGT can achieve improved bounds on dynamic regret without assuming gradient boundedness. Experimental results on logistic regression tasks confirm the effectiveness of TV-HSGT in dynamic and resource-constrained environments.