# Named Entity Recognition with Partially Annotated Training Data

arXiv: https://arxiv.org/abs/1909.09270 · HTML: https://mlsanity.com/paper/1909.09270/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 27 of 27)

- [Learning from Noisy Labels for Entity-Centric Information Extraction](https://mlsanity.com/paper/2104.08656/) (2021)
- [Noisy-Labeled NER with Confidence Estimation](https://mlsanity.com/paper/2104.04318/) (2021)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning](https://mlsanity.com/paper/2204.09589/) (2022)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Predictive Inference with Weak Supervision](https://mlsanity.com/paper/2201.08315/) (2022)
- [Rethinking Negative Sampling for Handling Missing Entity Annotations](https://mlsanity.com/paper/2108.11607/) (2021)
- [Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning](https://mlsanity.com/paper/2110.04429/) (2021)
- [Partially Supervised Named Entity Recognition via the Expected Entity Ratio Loss](https://mlsanity.com/paper/2108.07216/) (2021)
- [Improved Latent Tree Induction with Distant Supervision via Span Constraints](https://mlsanity.com/paper/2109.05112/) (2021)
- [SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalization](https://mlsanity.com/paper/2202.13100/) (2022)
- [Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training](https://mlsanity.com/paper/2305.12634/) (2023)
- [Partial Annotation Learning for Biomedical Entity Recognition](https://mlsanity.com/paper/2305.13120/) (2023)
- [Better Sampling of Negatives for Distantly Supervised Named Entity Recognition](https://mlsanity.com/paper/2305.13142/) (2023)
- [DMNER: Biomedical Entity Recognition by Detection and Matching](https://mlsanity.com/paper/2306.15736/) (2023)
- [CebuaNER: A New Baseline Cebuano Named Entity Recognition Model](https://mlsanity.com/paper/2310.00679/) (2023)
- [Combining Supervised Learning and Reinforcement Learning for Multi-Label Classification Tasks with Partial Labels](https://mlsanity.com/paper/2406.16293/) (2024)
- [Imbalances in Neurosymbolic Learning: Characterization and Mitigating Strategies](https://mlsanity.com/paper/2407.10000/) (2024)
- [Integrating curation into scientific publishing to train AI models](https://mlsanity.com/paper/2310.20440/) (2023)
- [NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity Recognition](https://mlsanity.com/paper/2405.07609/) (2024)
- [SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization](https://mlsanity.com/paper/2409.06216/) (2024)
- [Few Dimensions are Enough: Fine-tuning BERT with Selected Dimensions Revealed Its Redundant Nature](https://mlsanity.com/paper/2504.04966/) (2025)
- [AdaK-NER: An Adaptive Top-K Approach for Named Entity Recognition with Incomplete Annotations](https://mlsanity.com/paper/2109.05233/) (2021)
- [Named Entity Recognition for Partially Annotated Datasets](https://mlsanity.com/paper/2204.09081/) (2022)
- [Taxonomy Expansion for Named Entity Recognition](https://mlsanity.com/paper/2305.13191/) (2023)
- [Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition](https://mlsanity.com/paper/2606.21890/) (2026)
- [Strong Multilingual Privacy Tagging at Encoder Speed](https://mlsanity.com/paper/2609.38630/) (2026)

## Abstract (arXiv)

Supervised machine learning assumes the availability of fully-labeled data, but in many cases, such as low-resource languages, the only data available is partially annotated. We study the problem of Named Entity Recognition (NER) with partially annotated training data in which a fraction of the named entities are labeled, and all other tokens, entities or otherwise, are labeled as non-entity by default. In order to train on this noisy dataset, we need to distinguish between the true and false negatives. To this end, we introduce a constraint-driven iterative algorithm that learns to detect false negatives in the noisy set and downweigh them, resulting in a weighted training set. With this set, we train a weighted NER model. We evaluate our algorithm with weighted variants of neural and non-neural NER models on data in 8 languages from several language and script families, showing strong ability to learn from partial data. Finally, to show real-world efficacy, we evaluate on a Bengali NER corpus annotated by non-speakers, outperforming the prior state-of-the-art by over 5 points F1.