# CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction

arXiv: https://arxiv.org/abs/2106.10855 · HTML: https://mlsanity.com/paper/2106.10855/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Contrastive Learning for Image Captioning](https://mlsanity.com/paper/1710.02534/) (2017)
- [RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information](https://mlsanity.com/paper/1812.04361/) (2018)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention](https://mlsanity.com/paper/1804.06987/) (2018)
- [Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings](https://mlsanity.com/paper/2102.01156/) (2021)
- [Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction](https://mlsanity.com/paper/1812.10604/) (2018)
- [Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1904.09331/) (2019)

## Built on by (most cited first, 13 of 13)

- [A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm](https://mlsanity.com/paper/2310.12244/) (2023)
- [Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection](https://mlsanity.com/paper/2207.05500/) (2022)
- [HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2202.13352/) (2022)
- [Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networks](https://mlsanity.com/paper/2210.05498/) (2022)
- [Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction](https://mlsanity.com/paper/2210.11800/) (2022)
- [Global and Local Hierarchy-aware Contrastive Framework for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2211.13873/) (2022)
- [PARE: A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2110.07415/) (2021)
- [Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision](https://mlsanity.com/paper/2205.08770/) (2022)
- [Semantic-aware Contrastive Learning for More Accurate Semantic Parsing](https://mlsanity.com/paper/2301.07919/) (2023)
- [Visually Grounded Commonsense Knowledge Acquisition](https://mlsanity.com/paper/2211.12054/) (2022)
- [Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction](https://mlsanity.com/paper/2310.18912/) (2023)
- [SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN](https://mlsanity.com/paper/2507.06895/) (2025)
- [Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction](https://mlsanity.com/paper/2510.18344/) (2025)

## Abstract (arXiv)

The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task. For the past decade, researchers apply the multi-instance learning (MIL) framework to find the most reliable feature from a bag of sentences. Although the pattern of MIL bags can greatly reduce DS noise, it fails to represent many other useful sentence features in the datasets. In many cases, these sentence features can only be acquired by extra sentence-level human annotation with heavy costs. Therefore, the performance of distantly supervised RE models is bounded. In this paper, we go beyond typical MIL framework and propose a novel contrastive instance learning (CIL) framework. Specifically, we regard the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previous methods on NYT10, GDS and KBP.