# A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition

arXiv: https://arxiv.org/abs/2305.12485 · HTML: https://mlsanity.com/paper/2305.12485/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [NLNL: Negative Learning for Noisy Labels](https://mlsanity.com/paper/1908.07387/) (2019)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [Progressive Identification of True Labels for Partial-Label Learning](https://mlsanity.com/paper/2002.08053/) (2020)
- [Provably Consistent Partial-Label Learning](https://mlsanity.com/paper/2007.08929/) (2020)
- [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018)
- [TENER: Adapting Transformer Encoder for Named Entity Recognition](https://mlsanity.com/paper/1911.04474/) (2019)
- [Leveraged Weighted Loss for Partial Label Learning](https://mlsanity.com/paper/2106.05731/) (2021)
- [SpanNER: Named Entity Re-/Recognition as Span Prediction](https://mlsanity.com/paper/2106.00641/) (2021)
- [Partial Label Learning with Self-Guided Retraining](https://mlsanity.com/paper/1902.03045/) (2019)
- [Coarse-to-Fine Pre-training for Named Entity Recognition](https://mlsanity.com/paper/2010.08210/) (2020)
- [GM-PLL: Graph Matching based Partial Label Learning](https://mlsanity.com/paper/1901.03073/) (2019)
- [MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective](https://mlsanity.com/paper/2204.04391/) (2022)
- [A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck](https://mlsanity.com/paper/2208.13017/) (2022)
- [Adversarial Learning for Chinese NER from Crowd Annotations](https://mlsanity.com/paper/1801.05147/) (2018)

## Built on by (most cited first, 3 of 3)

- [Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMs](https://mlsanity.com/paper/2406.01943/) (2024)
- [Tuning the Right Foundation Models is What you Need for Partial Label Learning](https://mlsanity.com/paper/2506.05027/) (2025)
- [Trimming the Risk: Towards Reliable Continuous Training for Deep Learning Inspection Systems](https://mlsanity.com/paper/2409.09108/) (2024)

## Abstract (arXiv)

Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets, which always obtain using crowdsourcing. However, it is hard to obtain a unified and correct label via majority voting from multiple annotators for NER due to the large labeling space and complexity of this task. To address this problem, we aim to utilize the original multi-annotator labels directly. Particularly, we propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER. This model learns a token- and content-dependent confidence via an Expectation-Maximization (EM) algorithm by minimizing empirical risk. The true posterior estimator and confidence estimator perform iteratively to update the true posterior and confidence respectively. We conduct extensive experimental results on both real-world and synthetic datasets, which show that our model can improve performance effectively compared with strong baselines.