mlsanityData to 7 Oct 2026

Parametric Contrastive Learning

arXiv 2107.12028 ↗First posted 26 Jul 2021cs.CVcited by 217

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose Parametric Contrastive Learning (PaCo) to tackle long-tailed recognition. Based on theoretical analysis, we observe supervised contrastive loss tends to bias on high-frequency classes and thus increases the difficulty of imbalanced learning. We introduce a set of parametric class-wise learnable centers to rebalance from an optimization perspective. Further, we analyze our PaCo loss under a balanced setting. Our analysis demonstrates that PaCo can adaptively enhance the intensity of pushing samples of the same class close as more samples are pulled together with their corresponding centers and benefit hard example learning. Experiments on long-tailed CIFAR, ImageNet, Places, and iNaturalist 2018 manifest the new state-of-the-art for long-tailed recognition. On full ImageNet, models trained with PaCo loss surpass supervised contrastive learning across various ResNet backbones, e.g., our ResNet-200 achieves 81.8% top-1 accuracy. Our code is available at https://github.com/dvlab-research/Parametric-Contrastive-Learning.

Key facts. Cited by 217 papers in the map (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 26 Jul 2021.

Builds on

42 of 55 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2020Supervised Contrastive Learningcited by 2,803
  12. 2016Pyramid Scene Parsing Networkcited by 2,472
30 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  3. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  9. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  10. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  11. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  12. 2018Path Aggregation Network for Instance Segmentationcited by 796
  13. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  14. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  15. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  16. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  17. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  18. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
  19. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  20. 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 270
  21. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  22. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  23. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  24. 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
  25. 2021Contrastive Learning with Stronger Augmentationscited by 100
  26. 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
  27. 2018Deep Imbalanced Learning for Face Recognition and Attribute Predictioncited by 90
  28. 2016Factors in Finetuning Deep Model for object detectioncited by 53
  29. 2018Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examplescited by 27
  30. 2020ELF: An Early-Exiting Framework for Long-Tailed Classificationcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 33; 2023: 64; 2024: 48; 2025: 42; 2026: 21.

217 papers.

Sort:

By area:

  1. 2021Deep Long-Tailed Learning: A Surveycited by 416
  2. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  3. 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 143
  4. 2022Long-Tailed Recognition via Weight Balancingcited by 107
  5. 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
  6. 2021The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classificationcited by 100
  7. 2023Generating images of rare concepts using pre-trained diffusion modelscited by 78
  8. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  9. 2022LPT: Long-tailed Prompt Tuning for Image Classificationcited by 69
  10. 2021VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognitioncited by 65
  11. 2022Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 63
  12. 2023Decoupled Kullback-Leibler Divergence Losscited by 58
38 more
  1. 2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
  2. 2021A Simple Long-Tailed Recognition Baseline via Vision-Language Modelcited by 49
  3. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  4. 2023Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitionscited by 43
  5. 2023Norm-guided latent space exploration for text-to-image generationcited by 43
  6. 2024Probabilistic Contrastive Learning for Long-Tailed Visual Recognitioncited by 42
  7. 2022Expanding Low-Density Latent Regions for Open-Set Object Detectioncited by 38
  8. 2022Benchmarking Omni-Vision Representation through the Lens of Visual Realmscited by 38
  9. 2023How Re-sampling Helps for Long-Tail Learning?cited by 36
  10. 2022Learning Imbalanced Data with Vision Transformerscited by 34
  11. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  12. 2022OpenCon: Open-world Contrastive Learningcited by 33
  13. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  14. 2024LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Contentcited by 33
  15. 2023ImbSAM: A Closer Look at Sharpness-Aware Minimization in Class-Imbalanced Recognitioncited by 31
  16. 2023CUDA: Curriculum of Data Augmentation for Long-Tailed Recognitioncited by 30
  17. 2022BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learningcited by 29
  18. 2022Escaping Saddle Points for Effective Generalization on Class-Imbalanced Datacited by 28
  19. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  20. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  21. 2024A Systematic Review on Long-Tailed Learningcited by 27
  22. 2022Contrastive Learning with Boosted Memorizationcited by 26
  23. 2023Subclass-balancing Contrastive Learning for Long-tailed Recognitioncited by 26
  24. 2022Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasetscited by 25
  25. 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
  26. 2023Contrastive Learning Is Spectral Clustering On Similarity Graphcited by 24
  27. 2023A Unified Perspective for Loss-Oriented Imbalanced Learning via Localizationcited by 24
  28. 2022Delving into Semantic Scale Imbalancecited by 22
  29. 2023Long-Tailed Partial Label Learning via Dynamic Rebalancingcited by 22
  30. 2023Use Your Head: Improving Long-Tail Video Recognitioncited by 22
  31. 2023Long-Tailed Visual Recognition via Self-Heterogeneous Integration with Knowledge Excavationcited by 22
  32. 2021UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detectioncited by 20
  33. 2022Balanced Product of Calibrated Experts for Long-Tailed Recognitioncited by 20
  34. 2023Improving Image Recognition by Retrieving from Web-Scale Image-Text Datacited by 20
  35. 2023Feature Fusion from Head to Tail for Long-Tailed Visual Recognitioncited by 20
  36. 2023MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognitioncited by 19
  37. 2023ConR: Contrastive Regularizer for Deep Imbalanced Regressioncited by 19
  38. 2024Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwardscited by 19

50 most cited of 217 All 217 (JSON)