# Theory of Decentralized Robust Kernel-Based Learning

arXiv: https://arxiv.org/abs/2506.05215 · HTML: https://mlsanity.com/paper/2506.05215/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 54 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [Distributed optimization over time-varying directed graphs](https://mlsanity.com/paper/1303.2289/) (2013)
- [Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates](https://mlsanity.com/paper/1305.5029/) (2013)
- [Distributed learning with regularized least squares](https://mlsanity.com/paper/1608.03339/) (2016)
- [COKE: Communication-Censored Decentralized Kernel Learning](https://mlsanity.com/paper/2001.10133/) (2020)
- [Statistical Optimality of Divide and Conquer Kernel-based Functional Linear Regression](https://mlsanity.com/paper/2211.10968/) (2022)
- [Decentralized Online Learning with Kernels](https://mlsanity.com/paper/1710.04062/) (2017)
- [Distributed filtered hyperinterpolation for noisy data on the sphere](https://mlsanity.com/paper/1910.02434/) (2019)
- [Robust Kernel-based Distribution Regression](https://mlsanity.com/paper/2104.10637/) (2021)
- [Distributed Randomized Gradient-Free Mirror Descent Algorithm for Constrained Optimization](https://mlsanity.com/paper/1903.04157/) (2019)
- [Distributed Learning with Dependent Samples](https://mlsanity.com/paper/2002.03757/) (2020)
- [Optimality of Robust Online Learning](https://mlsanity.com/paper/2304.10060/) (2023)
- [Optimal Rates of Distributed Regression with Imperfect Kernels](https://mlsanity.com/paper/2006.16744/) (2020)
- [A Framework of Learning Through Empirical Gain Maximization](https://mlsanity.com/paper/2009.14250/) (2020)
- [Distributed Gradient Descent for Functional Learning](https://mlsanity.com/paper/2305.07408/) (2023)

## Abstract (arXiv)

We propose a new decentralized robust kernel-based learning algorithm within the framework of reproducing kernel Hilbert spaces (RKHSs) by utilizing a networked system that can be represented as a connected graph. The robust loss function huaLσ induced by a windowing function W and a robustness scaling parameter σ>0 can encompass a broad spectrum of robust losses. Consequently, the proposed algorithm effectively provides a unified decentralized learning framework for robust regression, which fundamentally differs from the existing distributed robust kernel-based learning schemes, all of which are divide-and-conquer based. We rigorously establish a learning theory and offer comprehensive convergence analysis for the algorithm. We show each local robust estimator generated from the decentralized algorithm can be utilized to approximate the regression function. Based on kernel-based integral operator techniques, we derive general high confidence convergence bounds for the local approximating sequence in terms of the mean square distance, RKHS norm, and generalization error, respectively. Moreover, we provide rigorous selection rules for local sample size and show that, under properly selected step size and scaling parameter σ, the decentralized robust algorithm can achieve optimal learning rates (up to logarithmic factors) in both norms. The parameter σ is shown to be essential for enhancing robustness and ensuring favorable convergence behavior. The intrinsic connection among decentralization, sample selection, robustness of the algorithm, and its convergence is clearly reflected.