mlsanityData to 7 Oct 2026

Distilling Virtual Examples for Long-tailed Recognition

arXiv 2103.15042 ↗First posted 28 Mar 2021cs.CVcited by 72

Not on the map. Why?

Abstract (arXiv)

We tackle the long-tailed visual recognition problem from the knowledge distillation perspective by proposing a Distill the Virtual Examples (DiVE) method. Specifically, by treating the predictions of a teacher model as virtual examples, we prove that distilling from these virtual examples is equivalent to label distribution learning under certain constraints. We show that when the virtual example distribution becomes flatter than the original input distribution, the under-represented tail classes will receive significant improvements, which is crucial in long-tailed recognition. The proposed DiVE method can explicitly tune the virtual example distribution to become flat. Extensive experiments on three benchmark datasets, including the large-scale iNaturalist ones, justify that the proposed DiVE method can significantly outperform state-of-the-art methods. Furthermore, additional analyses and experiments verify the virtual example interpretation, and demonstrate the effectiveness of tailored designs in DiVE for long-tailed problems.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 28 Mar 2021.

Builds on

26 of 37 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  8. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  9. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  10. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  11. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  12. 2020Long-tail learning via logit adjustmentcited by 605
14 more
  1. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  2. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
  3. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  4. 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 270
  5. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  6. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  7. 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
  8. 2018Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learningcited by 174
  9. 2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
  10. 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
  11. 2014Label Distribution Learningcited by 122
  12. 2015Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networkscited by 115
  13. 2016Deep Label Distribution Learning with Label Ambiguitycited by 104
  14. 2020Background Splitting: Finding Rare Classes in a Sea of Backgroundcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 22; 2023: 16; 2024: 17; 2025: 10; 2026: 4.

72 papers.

Sort:

By area:

  1. 2021Deep Long-Tailed Learning: A Surveycited by 416
  2. 2022Long-Tailed Recognition via Weight Balancingcited by 107
  3. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  4. 2023DELTA: degradation-free fully test-time adaptationcited by 72
  5. 2022Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 63
  6. 2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
  7. 2024Probabilistic Contrastive Learning for Long-Tailed Visual Recognitioncited by 42
  8. 2022Invariant Feature Learning for Generalized Long-Tailed Classificationcited by 34
  9. 2022Learning Imbalanced Data with Vision Transformerscited by 34
  10. 2023ImbSAM: A Closer Look at Sharpness-Aware Minimization in Class-Imbalanced Recognitioncited by 31
  11. 2022BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learningcited by 29
  12. 2022Escaping Saddle Points for Effective Generalization on Class-Imbalanced Datacited by 28
38 more
  1. 2024A Systematic Review on Long-Tailed Learningcited by 27
  2. 2021Introspective Distillation for Robust Question Answeringcited by 25
  3. 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
  4. 2023A Unified Perspective for Loss-Oriented Imbalanced Learning via Localizationcited by 24
  5. 2022Delving into Semantic Scale Imbalancecited by 22
  6. 2022When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Methodcited by 21
  7. 2022Balanced Product of Calibrated Experts for Long-Tailed Recognitioncited by 20
  8. 2021Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classificationcited by 19
  9. 2022NICEST: Noisy Label Correction and Training for Robust Scene Graph Generationcited by 19
  10. 2022Relieving Long-tailed Instance Segmentation via Pairwise Class Balancecited by 18
  11. 2022Label-Noise Learning with Intrinsically Long-Tailed Datacited by 17
  12. 2022Billion-user Customer Lifetime Value Prediction: An Industrial-scale Solution from Kuaishoucited by 17
  13. 2023No One Left Behind: Improving the Worst Categories in Long-Tailed Learningcited by 14
  14. 2023NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 14
  15. 2025Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classificationcited by 14
  16. 2023Quantized Feature Distillation for Network Quantizationcited by 13
  17. 2022Transfer and Share: Semi-Supervised Learning from Long-Tailed Datacited by 9
  18. 2023Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifoldscited by 9
  19. 2023SuperDisco: Super-Class Discovery Improves Visual Recognition for the Long-Tailcited by 9
  20. 2024Learning from Neighbors: Category Extrapolation for Long-Tail Learningcited by 9
  21. 2022The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognitioncited by 8
  22. 2023Long-Tailed Question Answering in an Open Worldcited by 7
  23. 2023Flexible Distribution Alignment: Towards Long-tailed Semi-supervised Learning with Proper Calibrationcited by 7
  24. 2024VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognitioncited by 7
  25. 2022Inverse Image Frequency for Long-tailed Image Recognitioncited by 6
  26. 2024Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifoldscited by 6
  27. 2024Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Taskscited by 6
  28. 2024LTRL: Boosting Long-tail Recognition via Reflective Learningcited by 6
  29. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  30. 2022Compound Batch Normalization for Long-tailed Image Classificationcited by 5
  31. 2023Instance-based Max-margin for Practical Few-shot Recognitioncited by 5
  32. 2023Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learningcited by 5
  33. 2024Rectify the Regression Bias in Long-Tailed Object Detectioncited by 5
  34. 2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
  35. 2024Fractal Calibration for long-tailed object detectioncited by 5
  36. 2025LIFT+: Lightweight Fine-Tuning for Long-Tail Learningcited by 5
  37. 2022Rebalanced Siamese Contrastive Mining for Long-Tailed Recognitioncited by 4
  38. 2022ImDrug: A Benchmark for Deep Imbalanced Learning in AI-aided Drug Discoverycited by 3

50 most cited of 72 All 72 (JSON)