Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification
Not on the map. Why?
Abstract (arXiv)
Federated Knowledge Graphs Embedding learning (FKGE) encounters challenges in communication efficiency stemming from the considerable size of parameters and extensive communication rounds. However, existing FKGE methods only focus on reducing communication rounds by conducting multiple rounds of local training in each communication round, and ignore reducing the size of parameters transmitted within each communication round. To tackle the problem, we first find that universal reduction in embedding precision across all entities during compression can significantly impede convergence speed, underscoring the importance of maintaining embedding precision. We then propose bidirectional communication-efficient FedS based on Entity-Wise Top-K Sparsification strategy. During upload, clients dynamically identify and upload only the Top-K entity embeddings with the greater changes to the server. During download, the server first performs personalized embedding aggregation for each client. It then identifies and transmits the Top-K aggregated embeddings to each client. Besides, an Intermittent Synchronization Mechanism is used by FedS to mitigate negative effect of embedding inconsistency among shared entities of clients caused by heterogeneity of Federated Knowledge Graph. Extensive experiments across three datasets showcase that FedS significantly enhances communication efficiency with negligible (even no) performance degradation.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 19 Jun 2024.
Builds on
25 of 44 references.
References · 25
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
- 2016Complex Embeddings for Simple Link Predictioncited by 902
- 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
- 2017Word Translation Without Parallel Datacited by 487
- 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
- 2017Sparse Communication for Distributed Gradient Descentcited by 249
- 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
- 2021Personalized Federated Learning using Hypernetworkscited by 212
- 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
13 more
- 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
- 2023From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labelscited by 69
- 2020Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsificationcited by 53
- 2023Making Large Language Models Perform Better in Knowledge Graph Completioncited by 48
- 2022Parameter-Efficient and Student-Friendly Knowledge Distillationcited by 47
- 2020FedE: Embedding Knowledge Graphs in Federated Settingcited by 46
- 2021Differentially Private Federated Knowledge Graphs Embeddingcited by 45
- 2021DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learningcited by 41
- 2023Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearningcited by 40
- 2022Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregationcited by 18
- 2020DualDE: Dually Distilling Knowledge Graph Embedding for Faster and Cheaper Reasoningcited by 17
- 2022Wireless Quantized Federated Learning: A Joint Computation and Communication Designcited by 15
- 2024Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graphcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.