# Few Clean Instances Help Denoising Distant Supervision

arXiv: https://arxiv.org/abs/2209.06596 · HTML: https://mlsanity.com/paper/2209.06596/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [On the Accuracy of Influence Functions for Measuring Group Effects](https://mlsanity.com/paper/1905.13289/) (2019)
- [Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning](https://mlsanity.com/paper/2007.01293/) (2020)
- [Reinforcement Learning for Relation Classification from Noisy Data](https://mlsanity.com/paper/1808.08013/) (2018)
- [Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning](https://mlsanity.com/paper/1805.09927/) (2018)
- [Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions](https://mlsanity.com/paper/1904.00143/) (2019)
- [DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction](https://mlsanity.com/paper/1805.09929/) (2018)
- [Understanding Goal-Oriented Active Learning via Influence Functions](https://mlsanity.com/paper/1905.13183/) (2019)
- [SEE: Syntax-aware Entity Embedding for Neural Relation Extraction](https://mlsanity.com/paper/1801.03603/) (2018)
- [Meta-Learning for Neural Relation Classification with Distant Supervision](https://mlsanity.com/paper/2010.13544/) (2020)

## Abstract (arXiv)

Existing distantly supervised relation extractors usually rely on noisy data for both model training and evaluation, which may lead to garbage-in-garbage-out systems. To alleviate the problem, we study whether a small clean dataset could help improve the quality of distantly supervised models. We show that besides getting a more convincing evaluation of models, a small clean dataset also helps us to build more robust denoising models. Specifically, we propose a new criterion for clean instance selection based on influence functions. It collects sample-level evidence for recognizing good instances (which is more informative than loss-level evidence). We also propose a teacher-student mechanism for controlling purity of intermediate results when bootstrapping the clean set. The whole approach is model-agnostic and demonstrates strong performances on both denoising real (NYT) and synthetic noisy datasets.