mlsanityData to 14 Oct 2026

Identifying Incorrect Annotations in Multi-Label Classification Data

arXiv 2211.13895 ↗First posted 25 Nov 2022cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In multi-label classification, each example in a dataset may be annotated as belonging to one or more classes (or none of the classes). Example applications include image (or document) tagging where each possible tag either applies to a particular image (or document) or not. With many possible classes to consider, data annotators are likely to make errors when labeling such data in practice. Here we consider algorithms for finding mislabeled examples in multi-label classification datasets. We propose an extension of the Confident Learning framework to this setting, as well as a label quality score that ranks examples with label errors much higher than those which are correctly labeled. Both approaches can utilize any trained classifier. After demonstrating that our methodology empirically outperforms other algorithms for label error detection, we apply our approach to discover many label errors in the CelebA image tagging dataset.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 11 of its 19 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

11 of 19 references.

References · 11

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  5. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
  6. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  7. 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
  8. 2022Annotation Error Detection: Analyzing the Past and Present for a More Coherent Futurecited by 45
  9. 2019Identifying Mislabeled Instances in Classification Datasetscited by 26
  10. 2022Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantificationcited by 20
  11. 2022Detecting Label Errors in Token Classification Datacited by 12

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 3; 2025: 3.

12 papers.

  1. 2023Identifying Label Errors in Object Detection Datasets by Loss Inspectioncited by 16
  2. 2023Estimating label quality and errors in semantic segmentation data via any modelcited by 12
  3. 2023ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Datacited by 11
  4. 2025Diffusion Counterfactual Generation with Semantic Abductioncited by 10
  5. 2023Positive Label Is All You Need for Multi-Label Classificationcited by 7
  6. 2024Automatic register identification for the open web using multilingual deep learningcited by 4
  7. 2023An Empirical Study of Automated Mislabel Detection in Real World Vision Datasetscited by 3
  8. 2023Detecting Errors in a Numerical Response via any Regression Modelcited by 0
  9. 2024A quest through interconnected datasets: lessons from highly-cited ICASSP paperscited by 0
  10. 2024Data-Centric Learning Framework for Real-Time Detection of Aiming Beam in Fluorescence Lifetime Imaging Guided Surgerycited by 0
  11. 2025From Label Error Detection to Correction: A Modular Framework and Benchmark for Object Detection Datasetscited by 0
  12. 2025Detecting and Rectifying Noisy Labels: A Similarity-based Approachcited by 0