mlsanityData to 7 Oct 2026

Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training

arXiv 2109.05003 ↗First posted 10 Sep 2021cs.CL cs.LGcited by 41

Not on the map. Why?

Abstract (arXiv)

We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base. The biggest challenge of distantly-supervised NER is that the distant supervision may induce incomplete and noisy labels, rendering the straightforward application of supervised learning ineffective. In this paper, we propose (1) a noise-robust learning scheme comprised of a new loss function and a noisy label removal step, for training NER models on distantly-labeled data, and (2) a self-training method that uses contextualized augmentations created by pre-trained language models to improve the generalization ability of the NER model. On three benchmark datasets, our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 31 of its 51 references that are in the map. First posted on arXiv 10 Sep 2021.

Builds on

31 of 51 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  7. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  8. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  11. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  12. 2016Neural Architectures for Named Entity Recognitioncited by 548
19 more
  1. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  2. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  3. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  4. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  5. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
  6. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
  7. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  8. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  9. 2020BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervisioncited by 91
  10. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 75
  11. 2018Weakly-Supervised Neural Text Classificationcited by 66
  12. 2018Learning Named Entity Tagger using Domain-Specific Dictionarycited by 65
  13. 2019Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learningcited by 46
  14. 2020Named Entity Recognition without Labelled Data: A Weak Supervision Approachcited by 46
  15. 2018Neural Cross-Lingual Named Entity Recognition with Minimal Resourcescited by 44
  16. 2018Weakly-Supervised Hierarchical Text Classificationcited by 42
  17. 2017Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projectioncited by 36
  18. 2019Learning to Denoise Distantly-Labeled Data for Entity Typingcited by 24
  19. 2018SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue Systemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 12; 2023: 12; 2024: 8; 2025: 5; 2026: 2.

41 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  2. 2021Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NERcited by 38
  3. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  4. 2022Language Models as Inductive Reasonerscited by 32
  5. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
  6. 2022Domain-Specific NER via Retrieving Correlated Samplescited by 11
  7. 2023Traffic-Domain Video Question Answering with Automatic Captioningcited by 10
  8. 2022Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intentscited by 9
  9. 2022Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generationcited by 9
  10. 2023A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trendscited by 8
  11. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  12. 2023PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Trainingcited by 6
29 more
  1. 2021Simple Questions Generate Named Entity Recognition Datasetscited by 5
  2. 2022General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generationcited by 5
  3. 2024Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approachcited by 5
  4. 2024CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learningcited by 5
  5. 2022PInKS: Preconditioned Commonsense Inference with Minimal Supervisioncited by 4
  6. 2023SANTA: Separate Strategies for Inaccurate and Incomplete Annotation Noise in Distantly-Supervised Named Entity Recognitioncited by 4
  7. 2023MProto: Multi-Prototype Network with Denoised Optimal Transport for Distantly Supervised Named Entity Recognitioncited by 4
  8. 2022Improving Distantly Supervised Relation Extraction by Natural Language Inferencecited by 3
  9. 2023Biomedical Named Entity Recognition via Dictionary-based Synonym Generalizationcited by 3
  10. 2023Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Trainingcited by 2
  11. 2023Better Sampling of Negatives for Distantly Supervised Named Entity Recognitioncited by 2
  12. 2023Distantly-Supervised Joint Extraction with Noise-Robust Learningcited by 2
  13. 2024Granular-ball Representation Learning for Deep CNN on Learning with Label Noisecited by 2
  14. 2025DynClean: Training Dynamics-based Label Cleaning for Distantly-Supervised Named Entity Recognitioncited by 2
  15. 2022Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representationscited by 1
  16. 2023Demonstration-based learning for few-shot biomedical named entity recognition under machine reading comprehensioncited by 1
  17. 2024ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Modelscited by 1
  18. 2022How to tackle an emerging topic? Combining strong and weak labels for Covid news NERcited by 0
  19. 2022MANER: Mask Augmented Named Entity Recognition for Extreme Low-Resource Languagescited by 0
  20. 2022On-the-fly Denoising for Data Augmentation in Natural Language Understandingcited by 0
  21. 2023Research on Named Entity Recognition in Improved transformer with R-Drop structurecited by 0
  22. 2023Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instancescited by 0
  23. 2024NanoNER: Named Entity Recognition for nanobiology using experts' knowledge and distant supervisioncited by 0
  24. 2024DistALANER: Distantly Supervised Active Learning Augmented Named Entity Recognition in the Open Source Software Ecosystemcited by 0
  25. 2025Covering Cracks in Content Moderation: Delexicalized Distant Supervision for Illicit Drug Jargon Detectioncited by 0
  26. 2025Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotationscited by 0
  27. 2025Comparing Approaches to Automatic Summarization in Less-Resourced Languagescited by 0
  28. 2026Learning Nested Named Entity Recognition from Flat Annotationscited by 0
  29. 2026Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation
  2. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models
  3. 2022kNN-NER: Named Entity Recognition with Nearest Neighbor Search
  4. 2024Rethinking Negative Instances for Generative Named Entity Recognition
  5. 2024Mix of Experts Language Model for Named Entity Recognition
  6. 2024ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models
  7. 2023RoPDA: Robust Prompt-based Data Augmentation for Low-Resource Named Entity Recognition
  8. 2026NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learning