# Semi-Supervised Contrastive Learning with Orthonormal Prototypes

arXiv: https://arxiv.org/abs/2512.07880 · HTML: https://mlsanity.com/paper/2512.07880/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 40 of its 61 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [Balanced Contrastive Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2207.09052/) (2022)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [A Large-Scale Car Dataset for Fine-Grained Categorization and Verification](https://mlsanity.com/paper/1506.08959/) (2015)
- [Boosting Contrastive Self-Supervised Learning with False Negative Cancellation](https://mlsanity.com/paper/2011.11765/) (2020)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [Rethinking Minimal Sufficient Representation in Contrastive Learning](https://mlsanity.com/paper/2203.07004/) (2022)
- [Class-Aware Contrastive Semi-Supervised Learning](https://mlsanity.com/paper/2203.02261/) (2022)
- [Hyperbolic Contrastive Learning for Visual Representations beyond Objects](https://mlsanity.com/paper/2212.00653/) (2022)
- [Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentation](https://mlsanity.com/paper/2109.07407/) (2021)
- [Simple and Asymmetric Graph Contrastive Learning without Augmentations](https://mlsanity.com/paper/2310.18884/) (2023)
- [Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast](https://mlsanity.com/paper/2202.09346/) (2022)

## Abstract (arXiv)

Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, dimensional collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we first identify a critical learning-rate threshold, beyond which standard contrastive losses converge to collapsed solutions. Building on these insights, we propose CLOP, a novel semi-supervised loss function designed to prevent dimensional collapse by promoting the formation of orthogonal linear subspaces among class embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP improves performance in image classification and object detection tasks while also exhibiting greater stability across different learning rates and batch sizes.