# Denoising Distantly Supervised Named Entity Recognition via a Hypergeometric Probabilistic Model

arXiv: https://arxiv.org/abs/2106.09234 · HTML: https://mlsanity.com/paper/2106.09234/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Learning Named Entity Tagger using Domain-Specific Dictionary](https://mlsanity.com/paper/1809.03599/) (2018)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning](https://mlsanity.com/paper/1906.01378/) (2019)
- [Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks](https://mlsanity.com/paper/1906.03783/) (2019)
- [Neural Segmental Hypergraphs for Overlapping Mention Recognition](https://mlsanity.com/paper/1810.01817/) (2018)
- [CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases](https://mlsanity.com/paper/1610.08763/) (2016)
- [A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land?](https://mlsanity.com/paper/2004.12126/) (2020)
- [Transfer Learning Between Related Tasks Using Expected Label Proportions](https://mlsanity.com/paper/1909.00430/) (2019)

## Built on by (most cited first, 7 of 7)

- [Fine-grained Entity Typing via Label Reasoning](https://mlsanity.com/paper/2109.05744/) (2021)
- [Few-shot Named Entity Recognition with Self-describing Networks](https://mlsanity.com/paper/2203.12252/) (2022)
- [BIOS: An Algorithmically Generated Biomedical Knowledge Graph](https://mlsanity.com/paper/2203.09975/) (2022)
- [Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approach](https://mlsanity.com/paper/2402.14948/) (2024)
- [MProto: Multi-Prototype Network with Denoised Optimal Transport for Distantly Supervised Named Entity Recognition](https://mlsanity.com/paper/2310.08298/) (2023)
- [PromptORE -- A Novel Approach Towards Fully Unsupervised Relation Extraction](https://mlsanity.com/paper/2304.01209/) (2023)
- [DynClean: Training Dynamics-based Label Cleaning for Distantly-Supervised Named Entity Recognition](https://mlsanity.com/paper/2504.04616/) (2025)

## Abstract (arXiv)

Denoising is the essential step for distant supervision based named entity recognition. Previous denoising methods are mostly based on instance-level confidence statistics, which ignore the variety of the underlying noise distribution on different datasets and entity types. This makes them difficult to be adapted to high noise rate settings. In this paper, we propose Hypergeometric Learning (HGL), a denoising algorithm for distantly supervised NER that takes both noise distribution and instance-level confidence into consideration. Specifically, during neural network training, we naturally model the noise samples in each batch following a hypergeometric distribution parameterized by the noise-rate. Then each instance in the batch is regarded as either correct or noisy one according to its label confidence derived from previous training step, as well as the noise distribution in this sampled batch. Experiments show that HGL can effectively denoise the weakly-labeled data retrieved from distant supervision, and therefore results in significant improvements on the trained models.