# DualDE: Dually Distilling Knowledge Graph Embedding for Faster and Cheaper Reasoning

arXiv: https://arxiv.org/abs/2009.05912 · HTML: https://mlsanity.com/paper/2009.05912/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2020.

## Built on by (most cited first, 17 of 17)

- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Meta-Knowledge Transfer for Inductive Knowledge Graph Embedding](https://mlsanity.com/paper/2110.14170/) (2021)
- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [Towards Continual Knowledge Graph Embedding via Incremental Distillation](https://mlsanity.com/paper/2405.04453/) (2024)
- [GreenKGC: A Lightweight Knowledge Graph Completion Method](https://mlsanity.com/paper/2208.09137/) (2022)
- [Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding](https://mlsanity.com/paper/2302.01849/) (2023)
- [Graph-based Knowledge Distillation: A survey and experimental evaluation](https://mlsanity.com/paper/2302.14643/) (2023)
- [Random Entity Quantization for Parameter-Efficient Compositional Knowledge Graph Representation](https://mlsanity.com/paper/2310.15797/) (2023)
- [Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification](https://mlsanity.com/paper/2406.13225/) (2024)
- [Self-Correction Distillation for Structured Data Question Answering](https://mlsanity.com/paper/2511.07998/) (2025)
- [From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding](https://mlsanity.com/paper/2303.12816/) (2023)
- [Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation](https://mlsanity.com/paper/2408.05748/) (2024)
- [ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models](https://mlsanity.com/paper/2510.09711/) (2025)
- [Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding](https://mlsanity.com/paper/2206.02963/) (2022)
- [Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion](https://mlsanity.com/paper/2401.12997/) (2024)
- [Croppable Knowledge Graph Embedding](https://mlsanity.com/paper/2407.02779/) (2024)
- [Unlearning of Knowledge Graph Embedding via Preference Optimization](https://mlsanity.com/paper/2507.20566/) (2025)

## Abstract (arXiv)

Knowledge Graph Embedding (KGE) is a popular method for KG reasoning and training KGEs with higher dimension are usually preferred since they have better reasoning capability. However, high-dimensional KGEs pose huge challenges to storage and computing resources and are not suitable for resource-limited or time-constrained applications, for which faster and cheaper reasoning is necessary. To address this problem, we propose DualDE, a knowledge distillation method to build low-dimensional student KGE from pre-trained high-dimensional teacher KGE. DualDE considers the dual-influence between the teacher and the student. In DualDE, we propose a soft label evaluation mechanism to adaptively assign different soft label and hard label weights to different triples, and a two-stage distillation approach to improve the student's acceptance of the teacher. Our DualDE is general enough to be applied to various KGEs. Experimental results show that our method can successfully reduce the embedding parameters of a high-dimensional KGE by 7 times - 15 times and increase the inference speed by 2 times - 6 times while retaining a high performance. We also experimentally prove the effectiveness of our soft label evaluation mechanism and two-stage distillation approach via ablation study.