# Distributed Optimization with Streaming Data: A Temporal Weighting Perspective

arXiv: https://arxiv.org/abs/2608.09565 · HTML: https://mlsanity.com/paper/2608.09565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 30 of 30)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [Non-stationary Stochastic Optimization](https://mlsanity.com/paper/1307.5449/) (2013)
- [Distributed Learning in Wireless Networks: Recent Progress and Future Challenges](https://mlsanity.com/paper/2104.02151/) (2021)
- [Online Optimization : Competing with Dynamic Comparators](https://mlsanity.com/paper/1501.06225/) (2015)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [On the Linear Convergence of the ADMM in Decentralized Consensus Optimization](https://mlsanity.com/paper/1307.5561/) (2013)
- [Distributed Online Optimization in Dynamic Environments Using Mirror Descent](https://mlsanity.com/paper/1609.02845/) (2016)
- [Time-Varying Convex Optimization: Time-Structured Algorithms and Applications](https://mlsanity.com/paper/2006.08500/) (2020)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [A Class of Prediction-Correction Methods for Time-Varying Convex Optimization](https://mlsanity.com/paper/1509.05196/) (2015)
- [Federated Learning for Data Streams](https://mlsanity.com/paper/2301.01542/) (2023)
- [Optimization and Learning with Information Streams: Time-varying Algorithms and Applications](https://mlsanity.com/paper/1910.08123/) (2019)
- [Can Primal Methods Outperform Primal-dual Methods in Decentralized Dynamic Optimization?](https://mlsanity.com/paper/2003.00816/) (2020)
- [Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Framework](https://mlsanity.com/paper/2503.14353/) (2025)
- [Non-Coherent Over-the-Air Decentralized Gradient Descent](https://mlsanity.com/paper/2211.10777/) (2022)
- [Distributed Convex Optimization for Continuous-Time Dynamics with Time-Varying Cost Function](https://mlsanity.com/paper/1507.04878/) (2015)
- [Prediction-Correction Interior-Point Method for Time-Varying Convex Optimization](https://mlsanity.com/paper/1608.07544/) (2016)
- [Dynamic Filtering of Time-Varying Sparse Signals via l1 Minimization](https://mlsanity.com/paper/1507.06145/) (2015)
- [Prediction-Correction Algorithms for Time-Varying Constrained Optimization](https://mlsanity.com/paper/1611.03681/) (2016)
- [Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach](https://mlsanity.com/paper/2405.12046/) (2024)
- [Decentralized Prediction-Correction Methods for Networked Time-Varying Convex Optimization](https://mlsanity.com/paper/1602.01716/) (2016)
- [Biased Federated Learning under Wireless Heterogeneity](https://mlsanity.com/paper/2503.06078/) (2025)
- [Time-Varying Optimization for Streaming Data Via Temporal Weighting](https://mlsanity.com/paper/2510.13052/) (2025)
- [Nonlinear Optimization Filters for Stochastic Time-Varying Convex Optimization](https://mlsanity.com/paper/2301.13646/) (2023)
- [Decentralized Time-Varying Optimization for Streaming Data via Temporal Weighting](https://mlsanity.com/paper/2605.06971/) (2026)

## Abstract (arXiv)

Optimization theory is a widely used tool for intelligent decision-making. While classical optimization deals with fixed, time-invariant objective functions, many modern applications operate in dynamic environments where data arrive sequentially, and the learning objective evolves over time, often under decentralized data and communication constraints. Motivated by these trends, we study decentralized optimization from streaming data through a structured time-varying formulation in which the global objective is a temporally weighted average of losses observed across the network. We analyze multi-iteration decentralized first-order methods, including decentralized gradient descent. For strongly convex and smooth losses, we develop guarantees for the Euclidean-norm tracking error through a contraction-mapping viewpoint. The resulting bounds decompose the tracking error into a fixed-point tracking component and a bias term induced by decentralization and data heterogeneity. We specialize our analysis to uniform and exponentially discounted weights, as well as their finite-memory windowed counterparts. The bounds explicitly characterize the roles of the temporal weighting rule, per-step iteration budget, step size, and network connectivity. Uniform weighting yields a vanishing fixed-point tracking contribution of order mathcal O(1/t), whereas discounted and windowed strategies generally induce non-vanishing tracking floors governed by the discount factor and effective memory, respectively. In all cases, decentralization induces an additional non-zero bias floor under a constant step size. Numerical experiments illustrate the predicted trends.