mlsanityData to 14 Oct 2026

Self-Damaging Contrastive Learning

arXiv 2106.02990 ↗First posted 6 Jun 2021cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

The recent breakthrough achieved by contrastive learning accelerates the pace for deploying unsupervised training on real-world data applications. However, unlabeled data in reality is commonly imbalanced and shows a long-tail distribution, and it is unclear how robustly the latest contrastive learning methods could perform in the practical scenario. This paper proposes to explicitly tackle this challenge, via a principled framework called Self-Damaging Contrastive Learning (SDCLR), to automatically balance the representation learning without knowing the classes. Our main inspiration is drawn from the recent finding that deep models have difficult-to-memorize samples, and those may be exposed through network pruning. It is further natural to hypothesize that long-tail samples are also tougher for the model to learn well due to insufficient examples. Hence, the key innovation in SDCLR is to create a dynamic self-competitor model to contrast with the target model, which is a pruned version of the latter. During training, contrasting the two models will lead to adaptive online mining of the most easily forgotten samples for the current target model, and implicitly emphasize them more in the contrastive loss. Extensive experiments across multiple datasets and imbalance settings show that SDCLR significantly improves not only overall accuracies but also balancedness, in terms of linear evaluation on the full-shot and few-shot settings. Our code is available at: https://github.com/VITA-Group/SDCLR.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

42 of 62 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  12. 2019Contrastive Multiview Codingcited by 1,301
30 more
  1. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  2. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  4. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  5. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  6. 2017A Closer Look at Memorization in Deep Networkscited by 942
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  9. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
  10. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  11. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  13. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  14. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  15. 2020Are we done with ImageNet?cited by 333
  16. 2018Slimmable Neural Networkscited by 305
  17. 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 271
  18. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  19. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  20. 2015Cost Sensitive Learning of Deep Feature Representations from Imbalanced Datacited by 167
  21. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  22. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  23. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  24. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 118
  25. 2015Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networkscited by 115
  26. 2017Do Convolutional Neural Networks Learn Class Hierarchy?cited by 61
  27. 2018Deep Imbalanced Attribute Classification using Visual Attention Aggregationcited by 56
  28. 2019Stabilizing the Lottery Ticket Hypothesiscited by 37
  29. 2021Troubleshooting Blind Image Quality Models in the Wildcited by 14
  30. 2020I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptivelycited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 14; 2024: 13; 2025: 6; 2026: 3.

48 papers.

Sort:

By area:

  1. 2022RankSim: Ranking Similarity Regularization for Deep Imbalanced Regressioncited by 52
  2. 2022Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Studycited by 45
  3. 2023Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Datacited by 44
  4. 2023DREAM: Visual Decoding from Reversing Human Visual Systemcited by 44
  5. 2022Rethinking the Augmentation Module in Contrastive Learning: Learning Hierarchical Augmentation Invariance with Expanded Viewscited by 37
  6. 2021Improving Contrastive Learning on Imbalanced Seed Data via Open-World Samplingcited by 30
  7. 2024A Systematic Review on Long-Tailed Learningcited by 28
  8. 2022Contrastive Learning with Boosted Memorizationcited by 26
  9. 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
  10. 2023Class-Imbalanced Learning on Graphs: A Surveycited by 24
  11. 2022ImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classificationcited by 21
  12. 2023On the Effectiveness of Out-of-Distribution Data in Self-Supervised Long-Tail Learningcited by 20
36 more
  1. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 19
  2. 2023Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representationscited by 18
  3. 2023SYNAuG: Exploiting Synthetic Data for Data Imbalance Problemscited by 13
  4. 2024P2OT: Progressive Partial Optimal Transport for Deep Imbalanced Clusteringcited by 13
  5. 2023Towards Distribution-Agnostic Generalized Category Discoverycited by 12
  6. 2023Combating Representation Learning Disparity with Geometric Harmonizationcited by 11
  7. 2024A Survey of Deep Long-Tail Classification Advancementscited by 11
  8. 2022Contrastive Learning with Positive-Negative Frame Mask for Music Representationcited by 10
  9. 2022Self-consistent Reasoning For Solving Math Word Problemscited by 10
  10. 2024CDMAD: Class-Distribution-Mismatch-Aware Debiasing for Class-Imbalanced Semi-Supervised Learningcited by 10
  11. 2024Training Class-Imbalanced Diffusion Model Via Overlap Optimizationcited by 6
  12. 2022Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalancescited by 4
  13. 2023Single-View View Synthesis with Self-Rectified Pseudo-Stereocited by 4
  14. 2023Frequency-Aware Self-Supervised Long-Tailed Learningcited by 4
  15. 2023Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruningcited by 4
  16. 2022ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentationcited by 3
  17. 2024SP2OT: Semantic-Regularized Progressive Partial Optimal Transport for Imbalanced Clusteringcited by 2
  18. 2024Views Can Be Deceiving: Improved SSL Through Feature Space Augmentationcited by 2
  19. 2025Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Modelcited by 1
  20. 2025Maximally Useful and Minimally Redundant: The Key to Self Supervised Learning for Imbalanced Datacited by 1
  21. 2026IConE: Batch Independent Collapse Prevention for Self-Supervised Representation Learningcited by 1
  22. 2026A Self supervised learning framework for imbalanced medical imaging datasetscited by 1
  23. 2022Beyond Labels: Visual Representations for Bone Marrow Cell Morphology Recognitioncited by 0
  24. 2023How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers?cited by 0
  25. 2023Damage GAN: A Generative Model for Imbalanced Datacited by 0
  26. 2023NearbyPatchCL: Leveraging Nearby Patches for Self-Supervised Patch-Level Multi-Class Classification in Whole-Slide Imagescited by 0
  27. 2024Self-degraded contrastive domain adaptation for industrial fault diagnosis with bi-imbalanced datacited by 0
  28. 2024Rethinking Open-World Semi-Supervised Learning: Distribution Mismatch and Inductive Inferencecited by 0
  29. 2024Music Era Recognition Using Supervised Contrastive Learning and Artist Informationcited by 0
  30. 2024On the Generalization and Causal Explanation in Self-Supervised Learningcited by 0
  31. 2024Uncovering Capabilities of Model Pruning in Graph Contrastive Learningcited by 0
  32. 2024Dataset Awareness is not Enough: Implementing Sample-level Tail Encouragement in Long-tailed Self-supervised Learningcited by 0
  33. 2025Long-Tailed Learning for Generalized Category Discoverycited by 0
  34. 2025Unsupervised Contrastive Learning Using Out-Of-Distribution Data for Long-Tailed Datasetcited by 0
  35. 2025Exploring Active Learning for Semiconductor Defect Segmentationcited by 0
  36. 2026PreFIQs: Face Image Quality Is What Survives Pruningcited by 0