# A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages

arXiv: https://arxiv.org/abs/2609.18739 · HTML: https://mlsanity.com/paper/2609.18739/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [LLMaAA: Making Large Language Models as Active Annotators](https://mlsanity.com/paper/2310.19596/) (2023)
- [How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks](https://mlsanity.com/paper/2303.00293/) (2023)
- [Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark](https://mlsanity.com/paper/2311.09122/) (2023)
- [Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies](https://mlsanity.com/paper/2312.11779/) (2023)
- [Detecting Label Errors in Token Classification Data](https://mlsanity.com/paper/2210.03920/) (2022)
- [CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Dataset](https://mlsanity.com/paper/2310.16225/) (2023)
- [Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery](https://mlsanity.com/paper/2403.06097/) (2024)
- [Urdu Word Segmentation using Conditional Random Fields (CRFs)](https://mlsanity.com/paper/1806.05432/) (2018)
- [Annotation Errors and NER: A Study with OntoNotes 5.0](https://mlsanity.com/paper/2406.19172/) (2024)
- [CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset](https://mlsanity.com/paper/2408.12362/) (2024)
- [CoNLL#: Fine-grained Error Analysis and a Corrected Test Set for CoNLL-03 English](https://mlsanity.com/paper/2405.11865/) (2024)
- [An Experimental Study on Data Augmentation Techniques for Named Entity Recognition on Low-Resource Domains](https://mlsanity.com/paper/2411.14551/) (2024)
- [Enhancing NER Performance in Low-Resource Pakistani Languages using Cross-Lingual Data Augmentation](https://mlsanity.com/paper/2504.08792/) (2025)
- [Enhancing Sindhi Word Segmentation using Subword Representation Learning and Position-aware Self-attention](https://mlsanity.com/paper/2012.15079/) (2020)

## Abstract (arXiv)

Poor quality or noisy annotations in Named Entity Recognition (NER), as in any other NLP task, make it challenging to achieve state-of-the-art performance. In this paper, we present a multi-step framework to enhance the annotation quality of NER datasets by employing automated techniques. We propose a frequency-based iterative approach that leverages self-training and a dual-threshold mechanism to enhance inference confidence. Experimental evaluations on different NER datasets demonstrate significant improvements in NER performance with respect to the original datasets. This work further explores the potential of generative Large Language Models (LLMs) to perform NER for low-resource languages.