Fusing Multiple Algorithms for Heterogeneous Online Learning
Not on the map. Why?
Abstract (arXiv)
This study addresses the challenge of online learning in contexts where agents accumulate disparate data, face resource constraints, and use different local algorithms. This paper introduces the Switched Online Learning Algorithm (SOLA), designed to solve the heterogeneous online learning problem by amalgamating updates from diverse agents through a dynamic switching mechanism contingent upon their respective performance and available resources. We theoretically analyze the design of the selecting mechanism to ensure that the regret of SOLA is bounded. Our findings show that the number of changes in selection needs to be bounded by a parameter dependent on the performance of the different local algorithms. Additionally, two test cases are presented to emphasize the effectiveness of SOLA, first on an online linear regression problem and then on an online classification problem with the MNIST dataset.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 38 references that are in the map. First posted on arXiv 9 Dec 2023.
Builds on
17 of 38 references.
References · 17
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2021Federated Learning on Non-IID Data: A Surveycited by 346
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
- 2017Three Factors Influencing Minima in SGDcited by 273
- 2018Online Learning: A Comprehensive Surveycited by 205
- 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 156
- 2023Deep Model Fusion: A Surveycited by 91
- 2021Cross-Gradient Aggregation for Decentralized Learning from Non-IID datacited by 33
- 2022Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Datacited by 32
- 2015Adaptive Online Learningcited by 23
- 2018Online Learning with Inexact Proximal Online Gradient Descent Algorithmscited by 22
5 more
- 2019Combining Online Learning Guaranteescited by 22
- 2021Remarks on input to state stability of perturbed gradient flows, motivated by model-free feedback control learningcited by 8
- 2018A continuous-time analysis of distributed stochastic gradientcited by 4
- 2022Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisionscited by 4
- 2021A Contraction Theory Approach to Optimization Algorithms from Acceleration Flowscited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.