# From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding

arXiv: https://arxiv.org/abs/2303.12816 · HTML: https://mlsanity.com/paper/2303.12816/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [Why does deep and cheap learning work so well?](https://mlsanity.com/paper/1608.08225/) (2016)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization](https://mlsanity.com/paper/1808.04122/) (2018)
- [STransE: a novel embedding model of entities and relationships in knowledge bases](https://mlsanity.com/paper/1606.08140/) (2016)
- [DualDE: Dually Distilling Knowledge Graph Embedding for Faster and Cheaper Reasoning](https://mlsanity.com/paper/2009.05912/) (2020)
- [Efficient Non-Sampling Knowledge Graph Embedding](https://mlsanity.com/paper/2104.10796/) (2021)
- [Hyperbolic Geometry is Not Necessary: Lightweight Euclidean-Based Models for Low-Dimensional Knowledge Graph Embeddings](https://mlsanity.com/paper/2103.14930/) (2021)
- [GreenKGC: A Lightweight Knowledge Graph Completion Method](https://mlsanity.com/paper/2208.09137/) (2022)
- [Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings](https://mlsanity.com/paper/2201.00565/) (2022)
- [MetaKG: Meta-learning on Knowledge Graph for Cold-start Recommendation](https://mlsanity.com/paper/2202.03851/) (2022)
- [Local and Global GANs with Semantic-Aware Upsampling for Image Generation](https://mlsanity.com/paper/2203.00047/) (2022)
- [PIE: a Parameter and Inference Efficient Solution for Large Scale Knowledge Graph Embedding Reasoning](https://mlsanity.com/paper/2204.13957/) (2022)
- [ER: Equivariance Regularizer for Knowledge Graph Completion](https://mlsanity.com/paper/2206.12142/) (2022)

## Built on by (most cited first, 1 of 1)

- [Parameter-Efficient Transformer Embeddings](https://mlsanity.com/paper/2505.02266/) (2025)

## Abstract (arXiv)

Knowledge graph embedding (KGE) that maps entities and relations into vector representations is essential for downstream applications. Conventional KGE methods require high-dimensional representations to learn the complex structure of knowledge graph, but lead to oversized model parameters. Recent advances reduce parameters by low-dimensional entity representations, while developing techniques (e.g., knowledge distillation or reinvented representation forms) to compensate for reduced dimension. However, such operations introduce complicated computations and model designs that may not benefit large knowledge graphs. To seek a simple strategy to improve the parameter efficiency of conventional KGE models, we take inspiration from that deeper neural networks require exponentially fewer parameters to achieve expressiveness comparable to wider networks for compositional structures. We view all entity representations as a single-layer embedding network, and conventional KGE methods that adopt high-dimensional entity representations equal widening the embedding network to gain expressiveness. To achieve parameter efficiency, we instead propose a deeper embedding network for entity representations, i.e., a narrow entity embedding layer plus a multi-layer dimension lifting network (LiftNet). Experiments on three public datasets show that by integrating LiftNet, four conventional KGE methods with 16-dimensional representations achieve comparable link prediction accuracy as original models that adopt 512-dimensional representations, saving 68.4% to 96.9% parameters.