# Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning

arXiv: https://arxiv.org/abs/2302.02069 · HTML: https://mlsanity.com/paper/2302.02069/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 4 Feb 2023.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)
- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)
- [When Machine Unlearning Jeopardizes Privacy](https://mlsanity.com/paper/2005.02205/) (2020)
- [FedKD: Communication Efficient Federated Learning via Knowledge Distillation](https://mlsanity.com/paper/2108.13323/) (2021)
- [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [Federated Knowledge Distillation](https://mlsanity.com/paper/2011.02367/) (2020)
- [FedE: Embedding Knowledge Graphs in Federated Setting](https://mlsanity.com/paper/2010.12882/) (2020)
- [Differentially Private Federated Knowledge Graphs Embedding](https://mlsanity.com/paper/2105.07615/) (2021)
- [Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation](https://mlsanity.com/paper/2203.09553/) (2022)

## Built on by (most cited first, 30 of 40)

- [Machine Unlearning: Solutions and Challenges](https://mlsanity.com/paper/2308.07061/) (2023)
- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [On Effects of Steering Latent Representation for Large Language Model Unlearning](https://mlsanity.com/paper/2408.06223/) (2024)
- [SAFE: Machine Unlearning With Shard Graphs](https://mlsanity.com/paper/2304.13169/) (2023)
- [Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics](https://mlsanity.com/paper/2401.05146/) (2024)
- [Federated Distillation: A Survey](https://mlsanity.com/paper/2404.08564/) (2024)
- [FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning](https://mlsanity.com/paper/2309.10283/) (2023)
- [Distill to Delete: Unlearning in Graph Networks with Knowledge Distillation](https://mlsanity.com/paper/2309.16173/) (2023)
- [A Survey of Graph Unlearning](https://mlsanity.com/paper/2310.02164/) (2023)
- [Trustworthy Federated Learning: Privacy, Security, and Beyond](https://mlsanity.com/paper/2411.01583/) (2024)
- [Machine Unlearning for Recommendation Systems: An Insight](https://mlsanity.com/paper/2401.10942/) (2024)
- [A Comprehensive Data-centric Overview of Federated Graph Learning](https://mlsanity.com/paper/2507.16541/) (2025)
- [Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation](https://mlsanity.com/paper/2307.03363/) (2023)
- [Vertical Federated Unlearning via Backdoor Certification](https://mlsanity.com/paper/2412.11476/) (2024)
- [TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning](https://mlsanity.com/paper/2502.19770/) (2025)
- [Goldfish: An Efficient Federated Unlearning Framework](https://mlsanity.com/paper/2404.03180/) (2024)
- [Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification](https://mlsanity.com/paper/2406.13225/) (2024)
- [Unlearning for Federated Online Learning to Rank: A Reproducibility Study](https://mlsanity.com/paper/2505.12791/) (2025)
- [Efficient Knowledge Graph Unlearning with Zeroth-order Information](https://mlsanity.com/paper/2508.14013/) (2025)
- [Exploring Federated Unlearning: Review, Comparison, and Insights](https://mlsanity.com/paper/2310.19218/) (2023)
- [VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings](https://mlsanity.com/paper/2603.02435/) (2026)
- [FedV-KGQA: Multi-Hop Question Answering over Vertically Partitioned Knowledge Graphs](https://mlsanity.com/paper/2608.24846/) (2026)
- [FedMKGC: Privacy-Preserving Federated Multilingual Knowledge Graph Completion](https://mlsanity.com/paper/2312.10645/) (2023)
- [Federated Knowledge Graph Unlearning via Diffusion Model](https://mlsanity.com/paper/2403.08554/) (2024)
- [Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation](https://mlsanity.com/paper/2408.05748/) (2024)
- [Learn to Unlearn: Meta-Learning-Based Knowledge Graph Embedding Unlearning](https://mlsanity.com/paper/2412.00881/) (2024)
- [Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2506.22036/) (2025)
- [REMISVFU: Vertical Federated Unlearning via Representation Misdirection for Intermediate Output Feature](https://mlsanity.com/paper/2512.10348/) (2025)
- [Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglement](https://mlsanity.com/paper/2604.04030/) (2026)

## Abstract (arXiv)

Federated Learning (FL) recently emerges as a paradigm to train a global machine learning model across distributed clients without sharing raw data. Knowledge Graph (KG) embedding represents KGs in a continuous vector space, serving as the backbone of many knowledge-driven applications. As a promising combination, federated KG embedding can fully take advantage of knowledge learned from different clients while preserving the privacy of local data. However, realistic problems such as data heterogeneity and knowledge forgetting still remain to be concerned. In this paper, we propose FedLU, a novel FL framework for heterogeneous KG embedding learning and unlearning. To cope with the drift between local optimization and global convergence caused by data heterogeneity, we propose mutual knowledge distillation to transfer local knowledge to global, and absorb global knowledge back. Moreover, we present an unlearning method based on cognitive neuroscience, which combines retroactive interference and passive decay to erase specific knowledge from local clients and propagate to the global model by reusing knowledge distillation. We construct new datasets for assessing realistic performance of the state-of-the-arts. Extensive experiments show that FedLU achieves superior results in both link prediction and knowledge forgetting.