# Graph Component Contrastive Learning for Concept Relatedness Estimation

arXiv: https://arxiv.org/abs/2206.12556 · HTML: https://mlsanity.com/paper/2206.12556/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [How Powerful are Spectral Graph Neural Networks](https://mlsanity.com/paper/2205.11172/) (2022)
- [Decoupling the Depth and Scope of Graph Neural Networks](https://mlsanity.com/paper/2201.07858/) (2022)
- [Graph-based Semi-supervised Learning: A Comprehensive Review](https://mlsanity.com/paper/2102.13303/) (2021)
- [SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction](https://mlsanity.com/paper/2004.02438/) (2020)
- [COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning](https://mlsanity.com/paper/2206.04726/) (2022)
- [Unsupervised Text Generation by Learning from Search](https://mlsanity.com/paper/2007.08557/) (2020)
- [Semi-supervised Relation Extraction via Incremental Meta Self-Training](https://mlsanity.com/paper/2010.16410/) (2020)
- [Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading](https://mlsanity.com/paper/2010.01838/) (2020)
- [Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction](https://mlsanity.com/paper/2109.06415/) (2021)
- [HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction](https://mlsanity.com/paper/2205.02225/) (2022)
- [Text Revision by On-the-Fly Representation Optimization](https://mlsanity.com/paper/2204.07359/) (2022)
- [A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model](https://mlsanity.com/paper/2201.10707/) (2022)
- [Improving Question Generation With to the Point Context](https://mlsanity.com/paper/1910.06036/) (2019)
- [Graph-adaptive Rectified Linear Unit for Graph Neural Networks](https://mlsanity.com/paper/2202.06281/) (2022)
- [Text Relatedness Based on a Word Thesaurus](https://mlsanity.com/paper/1401.5699/) (2014)

## Built on by (most cited first, 6 of 6)

- [A Survey on Vision-Language-Action Models for Embodied AI](https://mlsanity.com/paper/2405.14093/) (2024)
- [MECCH: Metapath Context Convolution-based Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2211.12792/) (2022)
- [Astra: Efficient Transformer Architecture and Contrastive Dynamics Learning for Embodied Instruction Following](https://mlsanity.com/paper/2408.01147/) (2024)
- [VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder](https://mlsanity.com/paper/2307.00852/) (2023)
- [FedHGN: A Federated Framework for Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2305.09729/) (2023)
- [GS-Quant: Granular Semantic and Generative Structural Quantization for Knowledge Graph Completion](https://mlsanity.com/paper/2604.21649/) (2026)

## Abstract (arXiv)

Concept relatedness estimation (CRE) aims to determine whether two given concepts are related. Existing methods only consider the pairwise relationship between concepts, while overlooking the higher-order relationship that could be encoded in a concept-level graph structure. We discover that this underlying graph satisfies a set of intrinsic properties of CRE, including reflexivity, commutativity, and transitivity. In this paper, we formalize the CRE properties and introduce a graph structure named ConcreteGraph. To address the data scarcity issue in CRE, we introduce a novel data augmentation approach to sample new concept pairs from the graph. As it is intractable for data augmentation to fully capture the structural information of the ConcreteGraph due to a large amount of potential concept pairs, we further introduce a novel Graph Component Contrastive Learning framework to implicitly learn the complete structure of the ConcreteGraph. Empirical results on three datasets show significant improvement over the state-of-the-art model. Detailed ablation studies demonstrate that our proposed approach can effectively capture the high-order relationship among concepts.