SigCLR: Sigmoid Contrastive Learning of Visual Representations
Not on the map. Why?
Abstract (arXiv)
We propose SigCLR: Sigmoid Contrastive Learning of Visual Representations. SigCLR utilizes the logistic loss that only operates on pairs and does not require a global view as in the cross-entropy loss used in SimCLR. We show that logistic loss shows competitive performance on CIFAR-10, CIFAR-100, and Tiny-IN compared to other established SSL objectives. Our findings verify the importance of learnable bias as in the case of SigLUP, however, it requires a fixed temperature as in the SimCLR to excel. Overall, SigCLR is a promising replacement for the SimCLR which is ubiquitous and has shown tremendous success in various domains.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 22 Oct 2024.
Builds on
36 of 43 references.
References · 36
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
24 more
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2020Graph Contrastive Learning with Augmentationscited by 1,203
- 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
- 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
- 2017Large Batch Training of Convolutional Networkscited by 529
- 2021ResNet strikes back: An improved training procedure in timmcited by 365
- 2020Are we done with ImageNet?cited by 329
- 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
- 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
- 2020Whitening for Self-Supervised Representation Learningcited by 278
- 2022A Contrastive Framework for Neural Text Generationcited by 224
- 2020Representation Learning via Invariant Causal Mechanismscited by 197
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
- 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
- 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
- 2021Solo-learn: A Library of Self-supervised Methods for Visual Representation Learningcited by 116
- 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
- 2022Self-Supervised Learning with an Information Maximization Criterioncited by 38
- 2022Non-Contrastive Learning Meets Language-Image Pre-Trainingcited by 25
Built on by
Citing papers per year: 2025: 1.
1 paper.