# Fine-grained Contrastive Learning for Relation Extraction

arXiv: https://arxiv.org/abs/2205.12491 · HTML: https://mlsanity.com/paper/2205.12491/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [Distant Supervision for Relation Extraction beyond the Sentence Boundary](https://mlsanity.com/paper/1609.04873/) (2016)
- [Relation Classification via Recurrent Neural Network](https://mlsanity.com/paper/1508.01006/) (2015)
- [Question Answering on Freebase via Relation Extraction and Textual Evidence](https://mlsanity.com/paper/1603.00957/) (2016)
- [SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction](https://mlsanity.com/paper/2109.12093/) (2021)
- [Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2105.09543/) (2021)
- [Abstractified Multi-instance Learning (AMIL) for Biomedical Relation Extraction](https://mlsanity.com/paper/2110.12501/) (2021)
- [Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision](https://mlsanity.com/paper/2205.08770/) (2022)

## Built on by (most cited first, 7 of 7)

- [Multi-level Contrastive Learning for Script-based Character Understanding](https://mlsanity.com/paper/2310.13231/) (2023)
- [AI-native Memory: A Pathway from LLMs Towards AGI](https://mlsanity.com/paper/2406.18312/) (2024)
- [Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction](https://mlsanity.com/paper/2305.11029/) (2023)
- [An Overview of Distant Supervision for Relation Extraction with a Focus on Denoising and Pre-training Methods](https://mlsanity.com/paper/2207.08286/) (2022)
- [Towards Zero-shot Relation Extraction in Web Mining: A Multimodal Approach with Relative XML Path](https://mlsanity.com/paper/2305.13805/) (2023)
- [READ: Improving Relation Extraction from an ADversarial Perspective](https://mlsanity.com/paper/2404.02931/) (2024)
- [GapDNER: A Gap-Aware Grid Tagging Model for Discontinuous Named Entity Recognition](https://mlsanity.com/paper/2510.10927/) (2025)

## Abstract (arXiv)

Recent relation extraction (RE) works have shown encouraging improvements by conducting contrastive learning on silver labels generated by distant supervision before fine-tuning on gold labels. Existing methods typically assume all these silver labels are accurate and treat them equally; however, distant supervision is inevitably noisy -- some silver labels are more reliable than others. In this paper, we propose fine-grained contrastive learning (FineCL) for RE, which leverages fine-grained information about which silver labels are and are not noisy to improve the quality of learned relationship representations for RE. We first assess the quality of silver labels via a simple and automatic approach we call "learning order denoising," where we train a language model to learn these relations and record the order of learned training instances. We show that learning order largely corresponds to label accuracy -- early-learned silver labels have, on average, more accurate labels than later-learned silver labels. Then, during pre-training, we increase the weights of accurate labels within a novel contrastive learning objective. Experiments on several RE benchmarks show that FineCL makes consistent and significant performance gains over state-of-the-art methods.