# Probabilistic Contrastive Learning for Long-Tailed Visual Recognition

arXiv: https://arxiv.org/abs/2403.06726 · HTML: https://mlsanity.com/paper/2403.06726/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 96 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 30 of 73)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)

## Built on by (most cited first, 30 of 42)

- [An overview of domain-specific foundation model: key technologies, applications and challenges](https://mlsanity.com/paper/2409.04267/) (2024)
- [CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning](https://mlsanity.com/paper/2504.13820/) (2025)
- [Selective Vision-Language Subspace Projection for Few-shot CLIP](https://mlsanity.com/paper/2407.16977/) (2024)
- [Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition](https://mlsanity.com/paper/2410.06109/) (2024)
- [Long-tailed Recognition with Model Rebalancing](https://mlsanity.com/paper/2510.08177/) (2025)
- [VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition](https://mlsanity.com/paper/2408.16930/) (2024)
- [UniTTA: Unified Benchmark and Versatile Framework Towards Realistic Test-Time Adaptation](https://mlsanity.com/paper/2407.20080/) (2024)
- [Long-Tailed Out-of-Distribution Detection: Prioritizing Attention to Tail](https://mlsanity.com/paper/2408.06742/) (2024)
- [Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models](https://mlsanity.com/paper/2405.14271/) (2024)
- [VLMine: Long-Tail Data Mining with Vision Language Models](https://mlsanity.com/paper/2409.15486/) (2024)
- [SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM](https://mlsanity.com/paper/2412.13715/) (2024)
- [vMFCoOp: Towards Equilibrium on a Unified Hyperspherical Manifold for Prompting Biomedical VLMs](https://mlsanity.com/paper/2511.09540/) (2025)
- [Enhancing Features in Long-tailed Data Using Large Vision Model](https://mlsanity.com/paper/2504.10852/) (2025)
- [A Survey on Small Sample Imbalance Problem: Metrics, Feature Analysis, and Solutions](https://mlsanity.com/paper/2504.14800/) (2025)
- [FRET: Feature Redundancy Elimination for Test Time Adaptation](https://mlsanity.com/paper/2505.10641/) (2025)
- [Aligned Contrastive Loss for Long-Tailed Recognition](https://mlsanity.com/paper/2506.01071/) (2025)
- [BAPE: Learning an Explicit Bayes Classifier for Long-tailed Visual Recognition](https://mlsanity.com/paper/2506.23280/) (2025)
- [Subtyping Breast Lesions via Generative Augmentation based Long-tailed Recognition in Ultrasound](https://mlsanity.com/paper/2507.22568/) (2025)
- [Multiview Self-Representation Learning across Heterogeneous Views](https://mlsanity.com/paper/2602.04328/) (2026)
- [VMF-GOS: Geometry-guided virtual Outlier Synthesis for Long-Tailed OOD Detection](https://mlsanity.com/paper/2602.05415/) (2026)
- [Equilibrium contrastive learning for imbalanced image classification](https://mlsanity.com/paper/2602.09506/) (2026)
- [GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling](https://mlsanity.com/paper/2606.04516/) (2026)
- [One Image is All You Need: Agentic One-Shot Image Generation via Text-Based World Models for Long-Tail Spatial Perception](https://mlsanity.com/paper/2606.20764/) (2026)
- [Imbalanced Semi-Supervised Learning via Label Refinement and Threshold Adjustment](https://mlsanity.com/paper/2407.05370/) (2024)
- [SAU: A Dual-Branch Network to Enhance Long-Tailed Recognition via Generative Models](https://mlsanity.com/paper/2408.16273/) (2024)
- [SeMi: When Imbalanced Semi-Supervised Learning Meets Mining Hard Examples](https://mlsanity.com/paper/2501.06004/) (2025)
- [Divergence-Based Similarity Function for Multi-View Contrastive Learning](https://mlsanity.com/paper/2507.06560/) (2025)
- [Semantic-guided Fine-tuning of Foundation Model for Long-tailed Visual Recognition](https://mlsanity.com/paper/2507.12807/) (2025)
- [OptiCorNet: Optimizing Sequence-Based Context Correlation for Visual Place Recognition](https://mlsanity.com/paper/2507.14477/) (2025)
- [FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements](https://mlsanity.com/paper/2510.12132/) (2025)

## Abstract (arXiv)

Long-tailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Such imbalance issue considerably impairs the performance of standard supervised learning algorithms, which are mainly designed for balanced training sets. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. However, the performance of supervised contrastive learning is plagued by an inherent challenge: it necessitates sufficiently large batches of training data to construct contrastive pairs that cover all categories, yet this requirement is difficult to meet in the context of class-imbalanced data. To overcome this obstacle, we propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space, and samples contrastive pairs accordingly. In fact, estimating the distributions of all classes using features in a small batch, particularly for imbalanced data, is not feasible. Our key idea is to introduce a reasonable and simple assumption that the normalized features in contrastive learning follow a mixture of von Mises-Fisher (vMF) distributions on unit space, which brings two-fold benefits. First, the distribution parameters can be estimated using only the first sample moment, which can be efficiently computed in an online manner across different batches. Second, based on the estimated distribution, the vMF distribution allows us to sample an infinite number of contrastive pairs and derive a closed form of the expected contrastive loss for efficient optimization. Our code is available at https://github.com/LeapLabTHU/ProCo.