Stochastic Contrastive Learning
Not on the map. Why?
Abstract (arXiv)
While state-of-the-art contrastive Self-Supervised Learning (SSL) models produce results competitive with their supervised counterparts, they lack the ability to infer latent variables. In contrast, prescribed latent variable (LV) models enable attributing uncertainty, inducing task specific compression, and in general allow for more interpretable representations. In this work, we introduce LV approximations to large scale contrastive SSL models. We demonstrate that this addition improves downstream performance (resulting in 96.42% and 77.49% test top-1 fine-tuned performance on CIFAR10 and ImageNet respectively with a ResNet50) as well as producing highly compressed representations (588x reduction) that are useful for interpretability, classification and regression downstream tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 1 Oct 2021.
Builds on
21 of 24 references.
References · 21
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
9 more
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
- 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
- 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
- 2019On Variational Bounds of Mutual Informationcited by 550
- 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
- 2020Self-Supervised MultiModal Versatile Networkscited by 276
- 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
- 2018Implicit Reparameterization Gradientscited by 84
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 1.
2 papers.