Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature
Not on the map. Why?
Abstract (arXiv)
Random feature (RF) has been widely used for node consistency in decentralized kernel ridge regression (KRR). Currently, the consistency is guaranteed by imposing constraints on coefficients of features, necessitating that the random features on different nodes are identical. However, in many applications, data on different nodes varies significantly on the number or distribution, which calls for adaptive and data-dependent methods that generate different RFs. To tackle the essential difficulty, we propose a new decentralized KRR algorithm that pursues consensus on decision functions, which allows great flexibility and well adapts data on nodes. The convergence is rigorously given and the effectiveness is numerically verified: by capturing the characteristics of the data on each node, while maintaining the same communication costs as other methods, we achieved an average regression accuracy improvement of 25.5% across six real-world data sets.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 52 references that are in the map. First posted on arXiv 13 May 2024.
Builds on
18 of 52 references.
References · 18
- 2020Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyondcited by 109
- 2013Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Ratescited by 65
- 2020Ternary Compression for Communication-Efficient Federated Learningcited by 57
- 2017Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamicscited by 28
- 2020Distributed Kernel Ridge Regression with Communicationscited by 14
- 2016A Fundamental Tradeoff between Computation and Communication in Distributed Computingcited by 12
- 2017Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Featurescited by 11
- 2020Decentralised Learning with Random Features and Distributed Gradient Descentcited by 9
- 2021Distributed Online Learning with Multiple Kernelscited by 9
- 2019Random Fourier Features via Fast Surrogate Leverage Weighted Samplingcited by 8
- 2020COKE: Communication-Censored Decentralized Kernel Learningcited by 8
- 2017Decentralized Online Learning with Kernelscited by 7
6 more
- 2020Active Learning with Multiple Kernelscited by 6
- 2019On Sampling Random Features From Empirical Leverage Scores: Implementation and Theoretical Guaranteescited by 5
- 2016A Scalable Framework for Wireless Distributed Computingcited by 4
- 2017On Data-Dependent Random Features for Improved Generalization in Supervised Learningcited by 4
- 2018Data-dependent compression of random features for large-scale kernel approximationcited by 3
- 2022A Decentralized Framework for Kernel PCA with Projection Consensus Constraintscited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.