# HaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embedding

arXiv: https://arxiv.org/abs/2305.10563 · HTML: https://mlsanity.com/paper/2305.10563/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 17 May 2023.

## Builds on (most cited first, 30 of 31)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Contrastive Learning with Adversarial Examples](https://mlsanity.com/paper/2010.12050/) (2020)
- [SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models](https://mlsanity.com/paper/2203.02167/) (2022)
- [ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning](https://mlsanity.com/paper/2110.02027/) (2021)
- [Knowledge Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2205.00976/) (2022)
- [PairRE: Knowledge Graph Embeddings via Paired Relation Vectors](https://mlsanity.com/paper/2011.03798/) (2020)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding](https://mlsanity.com/paper/1812.06410/) (2018)
- [Structure Aware Negative Sampling in Knowledge Graphs](https://mlsanity.com/paper/2009.11355/) (2020)
- [TempoQR: Temporal Question Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2112.05785/) (2021)
- [Relation Embedding with Dihedral Group in Knowledge Graph](https://mlsanity.com/paper/1906.00687/) (2019)
- [Joint Language Semantic and Structure Embedding for Knowledge Graph Completion](https://mlsanity.com/paper/2209.08721/) (2022)
- [KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completion](https://mlsanity.com/paper/2208.07622/) (2022)

## Built on by (most cited first, 3 of 3)

- [Negative Sampling in Knowledge Graph Representation Learning: A Review](https://mlsanity.com/paper/2402.19195/) (2024)
- [Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning](https://mlsanity.com/paper/2407.12703/) (2024)
- [REAL: Response Embedding-based Alignment for LLMs](https://mlsanity.com/paper/2409.17169/) (2024)

## Abstract (arXiv)

We consider a contrastive learning approach to knowledge graph embedding (KGE) via InfoNCE. For KGE, efficient learning relies on augmenting the training data with negative triples. However, most KGE works overlook the bias from generating the negative triples-false negative triples (factual triples missing from the knowledge graph). We argue that the generation of high-quality (i.e., hard) negative triples might lead to an increase in false negative triples. To mitigate the impact of false negative triples during the generation of hard negative triples, we propose the Hardness and Structure-aware (HaSa) contrastive KGE method, which alleviates the effect of false negative triples while generating the hard negative triples. Experiments show that HaSa improves the performance of InfoNCE-based KGE approaches and achieves state-of-the-art results in several metrics for WN18RR datasets and competitive results for FB15k-237 datasets compared to both classic and pre-trained LM-based KGE methods.