# Sem@K: Is my knowledge graph embedding model semantic-aware?

arXiv: https://arxiv.org/abs/2301.05601 · HTML: https://mlsanity.com/paper/2301.05601/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 84 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 30 of 41)

- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs](https://mlsanity.com/paper/1906.01195/) (2019)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [Knowledge Graph Embedding for Link Prediction: A Comparative Analysis](https://mlsanity.com/paper/2002.00819/) (2020)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [Analogical Inference for Multi-Relational Embeddings](https://mlsanity.com/paper/1705.02426/) (2017)
- [InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions](https://mlsanity.com/paper/1911.00219/) (2019)
- [Knowledge Base Completion: Baselines Strike Back](https://mlsanity.com/paper/1705.10744/) (2017)
- [Improving Knowledge Graph Embedding Using Simple Constraints](https://mlsanity.com/paper/1805.02408/) (2018)
- [Type-Constrained Representation Learning in Knowledge Graphs](https://mlsanity.com/paper/1508.02593/) (2015)
- [TransG : A Generative Mixture Model for Knowledge Graph Embedding](https://mlsanity.com/paper/1509.05488/) (2015)
- [Differentiating Concepts and Instances for Knowledge Graph Embedding](https://mlsanity.com/paper/1811.04588/) (2018)
- [Relation Embedding with Dihedral Group in Knowledge Graph](https://mlsanity.com/paper/1906.00687/) (2019)

## Built on by (most cited first, 10 of 10)

- [Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction](https://mlsanity.com/paper/2303.00286/) (2023)
- [Schema First! Learn Versatile Knowledge Graph Embeddings by Capturing Semantics with MASCHInE](https://mlsanity.com/paper/2306.03659/) (2023)
- [Towards Semantically Enriched Embeddings for Knowledge Graph Completion](https://mlsanity.com/paper/2308.00081/) (2023)
- [PyGraft: Configurable Generation of Synthetic Schemas and Knowledge Graphs at Your Fingertips](https://mlsanity.com/paper/2309.03685/) (2023)
- [Do Similar Entities have Similar Embeddings?](https://mlsanity.com/paper/2312.10370/) (2023)
- [On Large-scale Evaluation of Embedding Models for Knowledge Graph Completion](https://mlsanity.com/paper/2504.08970/) (2025)
- [Return of the Schema: Building Complete Datasets for Machine Learning and Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2602.14795/) (2026)
- [Improving Knowledge Graph Embeddings through Contrastive Learning with Negative Statements](https://mlsanity.com/paper/2510.11868/) (2025)
- [Beyond Accuracy: Measuring Logical Compliance of Predictive Models](https://mlsanity.com/paper/2606.20208/) (2026)
- [A hierarchy of faithfulness criteria for knowledge base completion](https://mlsanity.com/paper/2609.27863/) (2026)

## Abstract (arXiv)

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.