# Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction

arXiv: https://arxiv.org/abs/2312.12021 · HTML: https://mlsanity.com/paper/2312.12021/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [A Survey on Multi-view Learning](https://mlsanity.com/paper/1304.5634/) (2013)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.03659/) (2021)
- [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020)
- [FLEX: Unifying Evaluation for Few-Shot NLP](https://mlsanity.com/paper/2107.07170/) (2021)
- [Towards Representation Alignment and Uniformity in Collaborative Filtering](https://mlsanity.com/paper/2206.12811/) (2022)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs](https://mlsanity.com/paper/2007.02387/) (2020)
- [Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification](https://mlsanity.com/paper/1906.06678/) (2019)
- [Exploring Task Difficulty for Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.05473/) (2021)
- [MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction](https://mlsanity.com/paper/2109.04108/) (2021)
- [Entity Concept-enhanced Few-shot Relation Extraction](https://mlsanity.com/paper/2106.02401/) (2021)

## Built on by (most cited first, 5 of 5)

- [Automated Construction of Theme-specific Knowledge Graphs](https://mlsanity.com/paper/2404.19146/) (2024)
- [Few-Shot, No Problem: Descriptive Continual Relation Extraction](https://mlsanity.com/paper/2502.20596/) (2025)
- [WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Prompting](https://mlsanity.com/paper/2505.13944/) (2025)
- [Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extraction](https://mlsanity.com/paper/2601.20803/) (2026)
- [Few-shot Open Relation Extraction with Gaussian Prototype and Adaptive Margin](https://mlsanity.com/paper/2410.20320/) (2024)

## Abstract (arXiv)

Few-shot Relation Extraction (FSRE) aims to extract relational facts from a sparse set of labeled corpora. Recent studies have shown promising results in FSRE by employing Pre-trained Language Models (PLMs) within the framework of supervised contrastive learning, which considers both instances and label facts. However, how to effectively harness massive instance-label pairs to encompass the learned representation with semantic richness in this learning paradigm is not fully explored. To address this gap, we introduce a novel synergistic anchored contrastive pre-training framework. This framework is motivated by the insight that the diverse viewpoints conveyed through instance-label pairs capture incomplete yet complementary intrinsic textual semantics. Specifically, our framework involves a symmetrical contrastive objective that encompasses both sentence-anchored and label-anchored contrastive losses. By combining these two losses, the model establishes a robust and uniform representation space. This space effectively captures the reciprocal alignment of feature distributions among instances and relational facts, simultaneously enhancing the maximization of mutual information across diverse perspectives within the same relation. Experimental results demonstrate that our framework achieves significant performance enhancements compared to baseline models in downstream FSRE tasks. Furthermore, our approach exhibits superior adaptability to handle the challenges of domain shift and zero-shot relation extraction. Our code is available online at https://github.com/AONE-NLP/FSRE-SaCon.