# Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature

arXiv: https://arxiv.org/abs/2405.07791 · HTML: https://mlsanity.com/paper/2405.07791/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 18 of 18)

- [Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond](https://mlsanity.com/paper/2004.11154/) (2020)
- [Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates](https://mlsanity.com/paper/1305.5029/) (2013)
- [Ternary Compression for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2003.03564/) (2020)
- [Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics](https://mlsanity.com/paper/1712.09983/) (2017)
- [Distributed Kernel Ridge Regression with Communications](https://mlsanity.com/paper/2003.12210/) (2020)
- [A Fundamental Tradeoff between Computation and Communication in Distributed Computing](https://mlsanity.com/paper/1604.07086/) (2016)
- [Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Features](https://mlsanity.com/paper/1703.08131/) (2017)
- [Decentralised Learning with Random Features and Distributed Gradient Descent](https://mlsanity.com/paper/2007.00360/) (2020)
- [Distributed Online Learning with Multiple Kernels](https://mlsanity.com/paper/2102.12733/) (2021)
- [Random Fourier Features via Fast Surrogate Leverage Weighted Sampling](https://mlsanity.com/paper/1911.09158/) (2019)
- [COKE: Communication-Censored Decentralized Kernel Learning](https://mlsanity.com/paper/2001.10133/) (2020)
- [Decentralized Online Learning with Kernels](https://mlsanity.com/paper/1710.04062/) (2017)
- [Active Learning with Multiple Kernels](https://mlsanity.com/paper/2005.03188/) (2020)
- [On Sampling Random Features From Empirical Leverage Scores: Implementation and Theoretical Guarantees](https://mlsanity.com/paper/1903.08329/) (2019)
- [A Scalable Framework for Wireless Distributed Computing](https://mlsanity.com/paper/1608.05743/) (2016)
- [On Data-Dependent Random Features for Improved Generalization in Supervised Learning](https://mlsanity.com/paper/1712.07102/) (2017)
- [Data-dependent compression of random features for large-scale kernel approximation](https://mlsanity.com/paper/1810.04249/) (2018)
- [A Decentralized Framework for Kernel PCA with Projection Consensus Constraints](https://mlsanity.com/paper/2211.15953/) (2022)

## Built on by (most cited first, 1 of 1)

- [MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes](https://mlsanity.com/paper/2410.08557/) (2024)

## Abstract (arXiv)

Random feature (RF) has been widely used for node consistency in decentralized kernel ridge regression (KRR). Currently, the consistency is guaranteed by imposing constraints on coefficients of features, necessitating that the random features on different nodes are identical. However, in many applications, data on different nodes varies significantly on the number or distribution, which calls for adaptive and data-dependent methods that generate different RFs. To tackle the essential difficulty, we propose a new decentralized KRR algorithm that pursues consensus on decision functions, which allows great flexibility and well adapts data on nodes. The convergence is rigorously given and the effectiveness is numerically verified: by capturing the characteristics of the data on each node, while maintaining the same communication costs as other methods, we achieved an average regression accuracy improvement of 25.5% across six real-world data sets.