# Distributed Online Learning with Multiple Kernels

arXiv: https://arxiv.org/abs/2102.12733 · HTML: https://mlsanity.com/paper/2102.12733/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 50 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 10 of 10)

- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics](https://mlsanity.com/paper/1712.09983/) (2017)
- [Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Features](https://mlsanity.com/paper/1703.08131/) (2017)
- [Toward Intelligent Vehicular Networks: A Machine Learning Framework](https://mlsanity.com/paper/1804.00338/) (2018)
- [Nonparametric Basis Pursuit via Sparse Kernel-based Learning](https://mlsanity.com/paper/1302.5449/) (2013)
- [Active Learning with Multiple Kernels](https://mlsanity.com/paper/2005.03188/) (2020)
- [Distributed Adaptive Learning with Multiple Kernels in Diffusion Networks](https://mlsanity.com/paper/1801.07087/) (2018)

## Built on by (most cited first, 9 of 9)

- [Tighter Regret Analysis and Optimization of Online Federated Learning](https://mlsanity.com/paper/2205.06491/) (2022)
- [Gossiped and Quantized Online Multi-Kernel Learning](https://mlsanity.com/paper/2301.09848/) (2023)
- [Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers](https://mlsanity.com/paper/2403.14716/) (2024)
- [QC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM](https://mlsanity.com/paper/2208.02777/) (2022)
- [Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel](https://mlsanity.com/paper/2309.08201/) (2023)
- [Robust, Online, and Adaptive Decentralized Gaussian Processes](https://mlsanity.com/paper/2509.18011/) (2025)
- [Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature](https://mlsanity.com/paper/2405.07791/) (2024)
- [Decentralized Online Ensembles of Gaussian Processes for Multi-Agent Systems](https://mlsanity.com/paper/2502.05301/) (2025)
- [ZK-HybridFL: Zero-Knowledge Proof-Enhanced Hybrid Ledger for Federated Learning](https://mlsanity.com/paper/2601.22302/) (2026)

## Abstract (arXiv)

We consider the problem of learning a nonlinear function over a network of learners in a fully decentralized fashion. Online learning is additionally assumed, where every learner receives continuous streaming data locally. This learning model is called a fully distributed online learning (or a fully decentralized online federated learning). For this model, we propose a novel learning framework with multiple kernels, which is named DOMKL. The proposed DOMKL is devised by harnessing the principles of an online alternating direction method of multipliers and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret, implying that every learner in the network can learn a common function which has a diminishing gap from the best function in hindsight. Our analysis also reveals that DOMKL yields the same asymptotic performance of the state-of-the-art centralized approach while keeping local data at edge learners. Via numerical tests with real datasets, we demonstrate the effectiveness of the proposed DOMKL on various online regression and time-series prediction tasks.