# Crowdsourcing Semantic Label Propagation in Relation Classification

arXiv: https://arxiv.org/abs/1809.00537 · HTML: https://mlsanity.com/paper/1809.00537/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2018.

## Built on by (most cited first, 6 of 6)

- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Don't Waste a Single Annotation: Improving Single-Label Classifiers Through Soft Labels](https://mlsanity.com/paper/2311.05265/) (2023)
- [Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science](https://mlsanity.com/paper/2203.07031/) (2022)
- [Can Humans Identify Domains?](https://mlsanity.com/paper/2404.01785/) (2024)
- [Mixture of Experts based Multi-task Supervise Learning from Crowds](https://mlsanity.com/paper/2407.13268/) (2024)
- [Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions](https://mlsanity.com/paper/2509.08217/) (2025)

## Abstract (arXiv)

Distant supervision is a popular method for performing relation extraction from text that is known to produce noisy labels. Most progress in relation extraction and classification has been made with crowdsourced corrections to distant-supervised labels, and there is evidence that indicates still more would be better. In this paper, we explore the problem of propagating human annotation signals gathered for open-domain relation classification through the CrowdTruth methodology for crowdsourcing, that captures ambiguity in annotations by measuring inter-annotator disagreement. Our approach propagates annotations to sentences that are similar in a low dimensional embedding space, expanding the number of labels by two orders of magnitude. Our experiments show significant improvement in a sentence-level multi-class relation classifier.