# Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering

arXiv: https://arxiv.org/abs/2305.17019 · HTML: https://mlsanity.com/paper/2305.17019/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings](https://mlsanity.com/paper/2002.05969/) (2020)
- [End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion](https://mlsanity.com/paper/1811.04441/) (2018)
- [MISC: A MIxed Strategy-Aware Model Integrating COMET for Emotional Support Conversation](https://mlsanity.com/paper/2203.13560/) (2022)
- [Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering](https://mlsanity.com/paper/1909.05311/) (2019)
- [Commonsense Knowledge Base Completion with Structural and Semantic Context](https://mlsanity.com/paper/1910.02915/) (2019)
- [Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering](https://mlsanity.com/paper/2006.09073/) (2020)
- [Learn to Explain Efficiently via Neural Logic Inductive Learning](https://mlsanity.com/paper/1910.02481/) (2019)
- [Inductive Learning on Commonsense Knowledge Graph Completion](https://mlsanity.com/paper/2009.09263/) (2020)
- [Neutral Utterances are Also Causes: Enhancing Conversational Causal Emotion Entailment with Social Commonsense Knowledge](https://mlsanity.com/paper/2205.00759/) (2022)
- [Commonsense mining as knowledge base completion? A study on the impact of novelty](https://mlsanity.com/paper/1804.09259/) (2018)
- [MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation](https://mlsanity.com/paper/2210.07570/) (2022)

## Built on by (most cited first, 4 of 4)

- [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://mlsanity.com/paper/2409.13731/) (2024)
- [Efficient Knowledge Infusion via KG-LLM Alignment](https://mlsanity.com/paper/2406.03746/) (2024)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [KNOWCOMP POKEMON Team at DialAM-2024: A Two-Stage Pipeline for Detecting Relations in Dialogical Argument Mining](https://mlsanity.com/paper/2407.19740/) (2024)

## Abstract (arXiv)

The nodes in the commonsense knowledge graph (CSKG) are normally represented by free-form short text (e.g., word or phrase). Different nodes may represent the same concept. This leads to the problems of edge sparsity and node redundancy, which challenges CSKG representation and completion. On the one hand, edge sparsity limits the performance of graph representation learning; On the other hand, node redundancy makes different nodes corresponding to the same concept have inconsistent relations with other nodes. To address the two problems, we propose a new CSKG completion framework based on Contrastive Pretraining and Node Clustering (CPNC). Contrastive Pretraining constructs positive and negative head-tail node pairs on CSKG and utilizes contrastive learning to obtain better semantic node representation. Node Clustering aggregates nodes with the same concept into a latent concept, assisting the task of CSKG completion. We evaluate our CPNC approach on two CSKG completion benchmarks (CN-100K and ATOMIC), where CPNC outperforms the state-of-the-art methods. Extensive experiments demonstrate that both Contrastive Pretraining and Node Clustering can significantly improve the performance of CSKG completion. The source code of CPNC is publicly available on urlhttps://github.com/NUSTM/CPNC.