# Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction

arXiv: https://arxiv.org/abs/1904.09331 · HTML: https://mlsanity.com/paper/1904.09331/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2106.10855/) (2021)
- [Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training](https://mlsanity.com/paper/2106.10835/) (2021)
- [Knowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2109.02099/) (2021)
- [Learning Stable Predictors from Weak Supervision under Distribution Shift](https://mlsanity.com/paper/2604.05002/) (2026)
- [Hierarchical Relation-Guided Type-Sentence Alignment for Long-Tail Relation Extraction with Distant Supervision](https://mlsanity.com/paper/2109.09036/) (2021)
- [From Consensus to Disagreement: Multi-Teacher Distillation for Semi-Supervised Relation Extraction](https://mlsanity.com/paper/2112.01048/) (2021)
- [Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotations](https://mlsanity.com/paper/2505.12454/) (2025)

## Abstract (arXiv)

In recent years there is a surge of interest in applying distant supervision (DS) to automatically generate training data for relation extraction (RE). In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution. Specifically, we found this problem commonly exists in real-world DS datasets, and without special handing, typical DS-RE models cannot automatically adapt to this shift, thus achieving deteriorated performance. To further validate our intuition, we develop a simple yet effective adaptation method for DS-trained models, bias adjustment, which updates models learned over the source domain (i.e., DS training set) with a label distribution estimated on the target domain (i.e., test set). Experiments demonstrate that bias adjustment achieves consistent performance gains on DS-trained models, especially on neural models, with an up to 23% relative F1 improvement, which verifies our assumptions. Our code and data can be found at urlhttps://github.com/INK-USC/shifted-label-distribution.