# Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

arXiv: https://arxiv.org/abs/2201.00565 · HTML: https://mlsanity.com/paper/2201.00565/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 3 Jan 2022.

## Builds on (most cited first, 25 of 25)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020)
- [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019)
- [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017)
- [End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion](https://mlsanity.com/paper/1811.04441/) (2018)
- [Multi-relational Poincar'e Graph Embeddings](https://mlsanity.com/paper/1905.09791/) (2019)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [Learning Intents behind Interactions with Knowledge Graph for Recommendation](https://mlsanity.com/paper/2102.07057/) (2021)
- [DGL-KE: Training Knowledge Graph Embeddings at Scale](https://mlsanity.com/paper/2004.08532/) (2020)
- [A Semantic Matching Energy Function for Learning with Multi-relational Data](https://mlsanity.com/paper/1301.3485/) (2013)
- [Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completion](https://mlsanity.com/paper/2011.05816/) (2020)
- [Probability Calibration for Knowledge Graph Embedding Models](https://mlsanity.com/paper/1912.10000/) (2019)
- [Efficient Non-Sampling Knowledge Graph Embedding](https://mlsanity.com/paper/2104.10796/) (2021)
- [Hyperbolic Geometry is Not Necessary: Lightweight Euclidean-Based Models for Low-Dimensional Knowledge Graph Embeddings](https://mlsanity.com/paper/2103.14930/) (2021)
- [Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis](https://mlsanity.com/paper/2104.04676/) (2021)

## Built on by (most cited first, 8 of 8)

- [Knowledge Graph Self-Supervised Rationalization for Recommendation](https://mlsanity.com/paper/2307.02759/) (2023)
- [KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completion](https://mlsanity.com/paper/2208.07622/) (2022)
- [Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs](https://mlsanity.com/paper/2410.12609/) (2024)
- [MuCo-KGC: Multi-Context-Aware Knowledge Graph Completion](https://mlsanity.com/paper/2503.03091/) (2025)
- [HaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embedding](https://mlsanity.com/paper/2305.10563/) (2023)
- [River of No Return: Graph Percolation Embeddings for Efficient Knowledge Graph Reasoning](https://mlsanity.com/paper/2305.09974/) (2023)
- [Can Persistent Homology provide an efficient alternative for Evaluation of Knowledge Graph Completion Methods?](https://mlsanity.com/paper/2301.12929/) (2023)
- [From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding](https://mlsanity.com/paper/2303.12816/) (2023)

## Abstract (arXiv)

Knowledge graph embedding (KGE) has shown great potential in automatic knowledge graph (KG) completion and knowledge-driven tasks. However, recent KGE models suffer from high training cost and large storage space, thus limiting their practicality in real-world applications. To address this challenge, based on the latest findings in the field of Contrastive Learning, we propose a novel KGE training framework called Hardness-aware Low-dimensional Embedding (HaLE). Instead of the traditional Negative Sampling, we design a new loss function based on query sampling that can balance two important training targets, Alignment and Uniformity. Furthermore, we analyze the hardness-aware ability of recent low-dimensional hyperbolic models and propose a lightweight hardness-aware activation mechanism. The experimental results show that in the limited training time, HaLE can effectively improve the performance and training speed of KGE models on five commonly-used datasets. After training just a few minutes, the HaLE-trained models are competitive compared to the state-of-the-art models in both low- and high-dimensional conditions.