mlsanityData to 7 Oct 2026

Label Refinery: Improving ImageNet Classification through Label Progression

arXiv 1805.02641 ↗First posted 7 May 2018cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Among the three main components (data, labels, and models) of any supervised learning system, data and models have been the main subjects of active research. However, studying labels and their properties has received very little attention. Current principles and paradigms of labeling impose several challenges to machine learning algorithms. Labels are often incomplete, ambiguous, and redundant. In this paper we study the effects of various properties of labels and introduce the Label Refinery: an iterative procedure that updates the ground truth labels after examining the entire dataset. We show significant gain using refined labels across a wide range of models. Using a Label Refinery improves the state-of-the-art top-1 accuracy of (1) AlexNet from 59.3 to 67.2, (2) MobileNet from 70.6 to 73.39, (3) MobileNet-0.25 from 50.6 to 55.59, (4) VGG19 from 72.7 to 75.46, and (5) Darknet19 from 72.9 to 74.47.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 20; 2023: 12; 2024: 7; 2025: 3; 2026: 1.

65 papers.

Sort:

By area:

  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  2. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  3. 2021Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillationcited by 104
  4. 2021Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Studycited by 50
  5. 2022Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimationcited by 43
  6. 2021Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillationcited by 35
  7. 2022How many Observations are Enough? Knowledge Distillation for Trajectory Forecastingcited by 35
  8. 2021Sliced Recursive Transformercited by 26
  9. 2023Teacher-Student Architecture for Knowledge Distillation: A Surveycited by 26
  10. 2022Teacher-Student Architecture for Knowledge Learning: A Surveycited by 23
  11. 2021Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillationcited by 22
  12. 2022Object Localization under Single Coarse Point Supervisioncited by 18
38 more
  1. 2021Self-Adaptive Training: Bridging Supervised and Self-Supervised Learningcited by 17
  2. 2021Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillationcited by 17
  3. 2022On the benefits of knowledge distillation for adversarial robustnesscited by 17
  4. 2021Energy-constrained Self-training for Unsupervised Domain Adaptationcited by 16
  5. 2021Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classificationcited by 16
  6. 2021Few-Shot Learning with a Strong Teachercited by 16
  7. 2023Exploiting Unlabelled Photos for Stronger Fine-Grained SBIRcited by 14
  8. 2022Generalized Knowledge Distillation via Relationship Matchingcited by 12
  9. 2022A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Taskscited by 12
  10. 2022Complementary Bi-directional Feature Compression for Indoor 360deg Semantic Segmentation with Self-distillationcited by 12
  11. 2022Efficient One Pass Self-distillation with Zipf's Label Smoothingcited by 12
  12. 2024KV Prediction for Improved Time to First Tokencited by 9
  13. 2023Physical Knowledge Enhanced Deep Neural Network for Sea Surface Temperature Predictioncited by 8
  14. 2021ISP Distillationcited by 7
  15. 2021A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learningcited by 7
  16. 2021Unsupervised Domain Adaptation for RF-based Gesture Recognitioncited by 7
  17. 2021Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learningcited by 7
  18. 2022Boosting the Adversarial Transferability of Surrogate Models with Dark Knowledgecited by 6
  19. 2022Benign Overfitting in Classification: Provably Counter Label Noise with Larger Modelscited by 5
  20. 2023Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrievalcited by 4
  21. 2024Edge-guided and Class-balanced Active Learning for Semantic Segmentation of Aerial Imagescited by 4
  22. 2021Improving Binary Neural Networks through Fully Utilizing Latent Weightscited by 3
  23. 2022Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detectioncited by 3
  24. 2021A New Training Framework for Deep Neural Networkcited by 2
  25. 2022Jointly Learning Knowledge Embedding and Neighborhood Consensus with Relational Knowledge Distillation for Entity Alignmentcited by 2
  26. 2022Reducing Flipping Errors in Deep Neural Networkscited by 2
  27. 2022Constraining Pseudo-label in Self-training Unsupervised Domain Adaptation with Energy-based Modelcited by 2
  28. 2023Knowledge Distillation from Single to Multi Labels: an Empirical Studycited by 2
  29. 2023Expediting Contrastive Language-Image Pretraining via Self-distilled Encoderscited by 2
  30. 2025Sketch Down the FLOPs: Towards Efficient Networks for Human Sketchcited by 2
  31. 2021Label Augmentation via Time-based Knowledge Distillation for Financial Anomaly Detectioncited by 1
  32. 2021Knowledge Distillation with Noisy Labels for Natural Language Understandingcited by 1
  33. 2022Selective Cross-Task Distillationcited by 1
  34. 2022Federated Semi-Supervised Domain Adaptation via Knowledge Transfercited by 1
  35. 2023A Scalable and Efficient Iterative Method for Copying Machine Learning Classifierscited by 1
  36. 2023TaxoKnow: Taxonomy as Prior Knowledge in the Loss Function of Multi-class Classificationcited by 1
  37. 2023Perception and Semantic Aware Regularization for Sequential Confidence Calibrationcited by 1
  38. 2023Temporal Knowledge Distillation for Time-Sensitive Financial Services Applicationscited by 1

50 most cited of 65 All 65 (JSON)