# Detecting Label Errors in Token Classification Data

arXiv: https://arxiv.org/abs/2210.03920 · HTML: https://mlsanity.com/paper/2210.03920/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 8 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://mlsanity.com/paper/1909.01441/) (2019)
- [Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future](https://mlsanity.com/paper/2206.02280/) (2022)
- [Identifying Mislabeled Instances in Classification Datasets](https://mlsanity.com/paper/1912.05283/) (2019)
- [An Instance-Dependent Simulation Framework for Learning with Label Noise](https://mlsanity.com/paper/2107.11413/) (2021)

## Built on by (most cited first, 12 of 12)

- [Identifying Incorrect Annotations in Multi-Label Classification Data](https://mlsanity.com/paper/2211.13895/) (2022)
- [Estimating label quality and errors in semantic segmentation data via any model](https://mlsanity.com/paper/2307.05080/) (2023)
- [ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data](https://mlsanity.com/paper/2309.00832/) (2023)
- [CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Dataset](https://mlsanity.com/paper/2310.16225/) (2023)
- [DCA-Bench: A Benchmark for Dataset Curation Agents](https://mlsanity.com/paper/2406.07275/) (2024)
- [CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset](https://mlsanity.com/paper/2408.12362/) (2024)
- [Comparison Study Between Token Classification and Sequence Classification In Text Classification](https://mlsanity.com/paper/2211.13899/) (2022)
- [Information Extraction from Documents: Question Answering vs Token Classification in real-world setups](https://mlsanity.com/paper/2304.10994/) (2023)
- [Detecting Errors in a Numerical Response via any Regression Model](https://mlsanity.com/paper/2305.16583/) (2023)
- [Detecting and Rectifying Noisy Labels: A Similarity-based Approach](https://mlsanity.com/paper/2509.23964/) (2025)
- [Revising RVL-CDIP: Quantifying Errors and Test-Train Overlap](https://mlsanity.com/paper/2606.31446/) (2026)
- [A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages](https://mlsanity.com/paper/2609.18739/) (2026)

## Abstract (arXiv)

Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis. Here we consider the task of finding sentences that contain label errors in token classification datasets. We study 11 different straightforward methods that score tokens/sentences based on the predicted class probabilities output by a (any) token classification model (trained via any procedure). In precision-recall evaluations based on real-world label errors in entity recognition data from CoNLL-2003, we identify a simple and effective method that consistently detects those sentences containing label errors when applied with different token classification models.