mlsanityData to 14 Oct 2026

Variational Self-Supervised Contrastive Learning Using Beta Divergence

arXiv 2312.00824 ↗First posted 5 Sep 2023cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Learning a discriminative semantic space using unlabelled and noisy data remains unaddressed in a multi-label setting. We present a contrastive self-supervised learning method which is robust to data noise, grounded in the domain of variational methods. The method (VCL) utilizes variational contrastive learning with beta-divergence to learn robustly from unlabelled datasets, including uncurated and noisy datasets. We demonstrate the effectiveness of the proposed method through rigorous experiments including linear evaluation and fine-tuning scenarios with multi-label datasets in the face understanding domain. In almost all tested scenarios, VCL surpasses the performance of state-of-the-art self-supervised methods, achieving a noteworthy increase in accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

37 of 49 references.

References · 37

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  9. 2016Colorful Image Colorizationcited by 1,114
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  12. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
25 more
  1. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  2. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 198
  3. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  4. 2017Representation Learning by Learning to Countcited by 120
  5. 2017Unsupervised Learning by Predicting Noisecited by 98
  6. 2018Boosting Self-Supervised Learning via Knowledge Transfercited by 92
  7. 2021When Does Contrastive Visual Representation Learning Work?cited by 92
  8. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  9. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  10. 2017Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagerycited by 56
  11. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  12. 2018Self-supervised learning of a facial attribute embedding from videocited by 45
  13. 2018Self-Supervised Feature Learning by Learning to Spot Artifactscited by 43
  14. 2017Improvements to context based self-supervised learningcited by 32
  15. 2019Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Functioncited by 26
  16. 2022Is Self-Supervised Learning More Robust Than Supervised Learning?cited by 17
  17. 2022TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learningcited by 12
  18. 2019Self-Supervised Learning of Face Representations for Video Face Clusteringcited by 11
  19. 2016Face Attribute Prediction Using Off-the-Shelf CNN Featurescited by 10
  20. 2021ScatSimCLR: self-supervised contrastive learning with pretext task regularization for small-scale datasetscited by 9
  21. 2022Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognitioncited by 9
  22. 2022CNLL: A Semi-supervised Approach For Continual Noisy Label Learningcited by 8
  23. 2022Bootstrapping the Relationship Between Images and Their Clean and Noisy Labelscited by 7
  24. 2020Relative Attribute Classification with Deep Rank SVMcited by 4
  25. 2016Are Facial Attributes Adversarially Robust?cited by 3

Built on by

Not cited yet.