# Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure

arXiv: https://arxiv.org/abs/2211.10738 · HTML: https://mlsanity.com/paper/2211.10738/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 95 references that are in the map.
- First posted on arXiv 19 Nov 2022.

## Builds on (most cited first, 30 of 59)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [Deep Learning on Graphs: A Survey](https://mlsanity.com/paper/1812.04202/) (2018)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Graph Contrastive Learning Automated](https://mlsanity.com/paper/2106.07594/) (2021)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [Adversarial Graph Augmentation to Improve Graph Contrastive Learning](https://mlsanity.com/paper/2106.05819/) (2021)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Large-Scale Representation Learning on Graphs via Bootstrapping](https://mlsanity.com/paper/2102.06514/) (2021)

## Built on by (most cited first, 30 of 43)

- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training](https://mlsanity.com/paper/2405.20978/) (2024)
- [DSformer: A Double Sampling Transformer for Multivariate Time Series Long-term Prediction](https://mlsanity.com/paper/2308.03274/) (2023)
- [Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2405.14170/) (2024)
- [FactCHD: Benchmarking Fact-Conflicting Hallucination Detection](https://mlsanity.com/paper/2310.12086/) (2023)
- [Deep Temporal Graph Clustering](https://mlsanity.com/paper/2305.10738/) (2023)
- [Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment](https://mlsanity.com/paper/2302.07491/) (2023)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable Missing](https://mlsanity.com/paper/2405.11333/) (2024)
- [Fast Continual Multi-View Clustering with Incomplete Views](https://mlsanity.com/paper/2306.02389/) (2023)
- [Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement](https://mlsanity.com/paper/2308.03177/) (2023)
- [Dual Test-time Training for Out-of-distribution Recommender System](https://mlsanity.com/paper/2407.15620/) (2024)
- [Continual Multimodal Knowledge Graph Construction](https://mlsanity.com/paper/2305.08698/) (2023)
- [Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network](https://mlsanity.com/paper/2412.15678/) (2024)
- [Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation](https://mlsanity.com/paper/2404.09468/) (2024)
- [Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning](https://mlsanity.com/paper/2405.16869/) (2024)
- [RARE: Robust Masked Graph Autoencoder](https://mlsanity.com/paper/2304.01507/) (2023)
- [Unleashing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2402.15444/) (2024)
- [SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoning](https://mlsanity.com/paper/2304.10297/) (2023)
- [TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classification](https://mlsanity.com/paper/2309.11845/) (2023)
- [RELIEF: Reinforcement Learning Empowered Graph Feature Prompt Tuning](https://mlsanity.com/paper/2408.03195/) (2024)
- [Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning](https://mlsanity.com/paper/2505.02639/) (2025)
- [Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning](https://mlsanity.com/paper/2307.03591/) (2023)
- [Local-Global History-aware Contrastive Learning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2312.01601/) (2023)
- [Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off](https://mlsanity.com/paper/2312.10329/) (2023)
- [Faithful and Consistent Graph Neural Network Explanations with Rationale Alignment](https://mlsanity.com/paper/2301.02791/) (2023)
- [Rethinking Regularization Methods for Knowledge Graph Completion](https://mlsanity.com/paper/2505.23442/) (2025)
- [Learning to Denoise Biomedical Knowledge Graph for Robust Molecular Interaction Prediction](https://mlsanity.com/paper/2312.06682/) (2023)
- [Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding](https://mlsanity.com/paper/2410.15702/) (2024)

## Abstract (arXiv)

Knowledge graph embedding (KGE) aims at learning powerful representations to benefit various artificial intelligence applications. Meanwhile, contrastive learning has been widely leveraged in graph learning as an effective mechanism to enhance the discriminative capacity of the learned representations. However, the complex structures of KG make it hard to construct appropriate contrastive pairs. Only a few attempts have integrated contrastive learning strategies with KGE. But, most of them rely on language models ( e.g., Bert) for contrastive pair construction instead of fully mining information underlying the graph structure, hindering expressive ability. Surprisingly, we find that the entities within a relational symmetrical structure are usually similar and correlated. To this end, we propose a knowledge graph contrastive learning framework based on relation-symmetrical structure, KGE-SymCL, which mines symmetrical structure information in KGs to enhance the discriminative ability of KGE models. Concretely, a plug-and-play approach is proposed by taking entities in the relation-symmetrical positions as positive pairs. Besides, a self-supervised alignment loss is designed to pull together positive pairs. Experimental results on link prediction and entity classification datasets demonstrate that our KGE-SymCL can be easily adopted to various KGE models for performance improvements. Moreover, extensive experiments show that our model could outperform other state-of-the-art baselines.