# PARE: A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction

arXiv: https://arxiv.org/abs/2110.07415 · HTML: https://mlsanity.com/paper/2110.07415/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction](https://mlsanity.com/paper/1909.13078/) (2019)
- [More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction](https://mlsanity.com/paper/2004.03186/) (2020)
- [RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information](https://mlsanity.com/paper/1812.04361/) (2018)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning](https://mlsanity.com/paper/1805.09927/) (2018)
- [CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2106.10855/) (2021)
- [Multilingual Knowledge Graph Completion with Joint Relation and Entity Alignment](https://mlsanity.com/paper/2104.08804/) (2021)
- [Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2105.09543/) (2021)
- [Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings](https://mlsanity.com/paper/2102.01156/) (2021)
- [DiS-ReX: A Multilingual Dataset for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2104.08655/) (2021)
- [Towards Semi-Supervised Learning for Deep Semantic Role Labeling](https://mlsanity.com/paper/1808.09543/) (2018)
- [From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension](https://mlsanity.com/paper/2012.04334/) (2020)

## Built on by (most cited first, 4 of 4)

- [ZGUL: Zero-shot Generalization to Unseen Languages using Multi-source Ensembling of Language Adapters](https://mlsanity.com/paper/2310.16393/) (2023)
- [Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction](https://mlsanity.com/paper/2310.18912/) (2023)
- [2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion](https://mlsanity.com/paper/2404.17122/) (2024)
- [SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN](https://mlsanity.com/paper/2507.06895/) (2025)

## Abstract (arXiv)

Neural models for distantly supervised relation extraction (DS-RE) encode each sentence in an entity-pair bag separately. These are then aggregated for bag-level relation prediction. Since, at encoding time, these approaches do not allow information to flow from other sentences in the bag, we believe that they do not utilize the available bag data to the fullest. In response, we explore a simple baseline approach (PARE) in which all sentences of a bag are concatenated into a passage of sentences, and encoded jointly using BERT. The contextual embeddings of tokens are aggregated using attention with the candidate relation as query -- this summary of whole passage predicts the candidate relation. We find that our simple baseline solution outperforms existing state-of-the-art DS-RE models in both monolingual and multilingual DS-RE datasets.