# LEMON: LanguagE ModeL for Negative Sampling of Knowledge Graph Embeddings

arXiv: https://arxiv.org/abs/2203.04703 · HTML: https://mlsanity.com/paper/2203.04703/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 9 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [On Sampling Strategies for Neural Network-based Collaborative Filtering](https://mlsanity.com/paper/1706.07881/) (2017)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding](https://mlsanity.com/paper/1812.06410/) (2018)
- [Incorporating GAN for Negative Sampling in Knowledge Representation Learning](https://mlsanity.com/paper/1809.11017/) (2018)
- [Structure Aware Negative Sampling in Knowledge Graphs](https://mlsanity.com/paper/2009.11355/) (2020)
- [Distributional Negative Sampling for Knowledge Base Completion](https://mlsanity.com/paper/1908.06178/) (2019)
- [MixKG: Mixing for harder negative samples in knowledge graph](https://mlsanity.com/paper/2202.09606/) (2022)

## Built on by (most cited first, 2 of 2)

- [Unsupervised Pretraining for Fact Verification by Language Model Distillation](https://mlsanity.com/paper/2309.16540/) (2023)
- [A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and Prospects](https://mlsanity.com/paper/2506.11012/) (2025)

## Abstract (arXiv)

Knowledge Graph Embedding models have become an important area of machine learning.Those models provide a latent representation of entities and relations in a knowledge graph which can then be used in downstream machine learning tasks such as link prediction. The learning process of such models can be performed by contrasting positive and negative triples. While all triples of a KG are considered positive, negative triples are usually not readily available. Therefore, the choice of the sampling method to obtain the negative triples play a crucial role in the performance and effectiveness of Knowledge Graph Embedding models. Most of the current methods fetch negative samples from a random distribution of entities in the underlying Knowledge Graph which also often includes meaningless triples. Other known methods use adversarial techniques or generative neural networks which consequently reduce the efficiency of the process. In this paper, we propose an approach for generating informative negative samples considering available complementary knowledge about entities. Particularly, Pre-trained Language Models are used to form neighborhood clusters by utilizing the distances between entities to obtain representations of symbolic entities via their textual information. Our comprehensive evaluations demonstrate the effectiveness of the proposed approach on benchmark Knowledge Graphs with textual information for the link prediction task.