mlsanityData to 7 Oct 2026

Noisy-Labeled NER with Confidence Estimation

arXiv 2104.04318 ↗First posted 9 Apr 2021cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Recent studies in deep learning have shown significant progress in named entity recognition (NER). Most existing works assume clean data annotation, yet a fundamental challenge in real-world scenarios is the large amount of noise from a variety of sources (e.g., pseudo, weak, or distant annotations). This work studies NER under a noisy labeled setting with calibrated confidence estimation. Based on empirical observations of different training dynamics of noisy and clean labels, we propose strategies for estimating confidence scores based on local and global independence assumptions. We partially marginalize out labels of low confidence with a CRF model. We further propose a calibration method for confidence scores based on the structure of entity labels. We integrate our approach into a self-training framework for boosting performance. Experiments in general noisy settings with four languages and distantly labeled settings demonstrate the effectiveness of our method. Our code can be found at https://github.com/liukun95/Noisy-NER-Confidence-Estimation

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 16 of its 34 references that are in the map. First posted on arXiv 9 Apr 2021.

Builds on

16 of 34 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  4. 2017A Closer Look at Memorization in Deep Networkscited by 936
  5. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  6. 2016Neural Architectures for Named Entity Recognitioncited by 548
  7. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  8. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  9. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  10. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  11. 2020BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervisioncited by 91
  12. 2018Learning Named Entity Tagger using Domain-Specific Dictionarycited by 65
4 more
  1. 2019Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learningcited by 46
  2. 2019Low-Resource Name Tagging Learned with Weakly Labeled Datacited by 32
  3. 2019Named Entity Recognition with Partially Annotated Training Datacited by 27
  4. 2018Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Datacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 8; 2024: 2; 2025: 1; 2026: 1.

28 papers.

Sort:

By area:

  1. 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 95
  2. 2022Generative Knowledge Graph Construction: A Reviewcited by 49
  3. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  4. 2021Gradient Imitation Reinforcement Learning for Low Resource Relation Extractioncited by 33
  5. 2022InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NERcited by 31
  6. 2021LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Promptingcited by 27
  7. 2023CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoningcited by 25
  8. 2022Ontology-enhanced Prompt-tuning for Few-shot Learningcited by 24
  9. 2022Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learningcited by 24
  10. 2022Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learningcited by 21
  11. 2024SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documentscited by 20
  12. 2022DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Populationcited by 17
16 more
  1. 2023One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NERcited by 11
  2. 2021Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understandingcited by 9
  3. 2022Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extractioncited by 7
  4. 2021Simple Questions Generate Named Entity Recognition Datasetscited by 5
  5. 2022Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Studycited by 5
  6. 2024Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approachcited by 5
  7. 2023Schema-adaptable Knowledge Graph Constructioncited by 4
  8. 2023Integrating Large Pre-trained Models into Multimodal Named Entity Recognition with Evidential Fusioncited by 4
  9. 2022Improve Event Extraction via Self-Training with Gradient Guidancecited by 3
  10. 2023Better Sampling of Negatives for Distantly Supervised Named Entity Recognitioncited by 2
  11. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  12. 2025Few Dimensions are Enough: Fine-tuning BERT with Selected Dimensions Revealed Its Redundant Naturecited by 1
  13. 2022Gradient Imitation Reinforcement Learning for General Low-Resource Information Extractioncited by 0
  14. 2023A Curriculum View of Robust Loss Functionscited by 0
  15. 2023Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instancescited by 0
  16. 2026Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labelscited by 0