mlsanityData to 14 Oct 2026

Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere

arXiv 2405.16460 ↗First posted 26 May 2024cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised contrastive learning has predominantly adopted deterministic methods, which are not suited for environments characterized by uncertainty and noise. This paper introduces a new perspective on incorporating uncertainty into contrastive learning by embedding representations within a spherical space, inspired by the von Mises-Fisher distribution (vMF). We introduce an unnormalized form of vMF and leverage the concentration parameter, kappa, as a direct, interpretable measure to quantify uncertainty explicitly. This approach not only provides a probabilistic interpretation of the embedding space but also offers a method to calibrate model confidence against varying levels of data corruption and characteristics. Our empirical results demonstrate that the estimated concentration parameter correlates strongly with the degree of unforeseen data corruption encountered at test time, enables failure analysis, and enhances existing out-of-distribution detection methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

43 of 56 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  12. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
31 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  3. 2019Contrastive Multiview Codingcited by 1,301
  4. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 737
  5. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  6. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  7. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  8. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  9. 2020Understanding the Behaviour of Contrastive Losscited by 398
  10. 2017Snapshot Ensembles: Train 1, get M for freecited by 350
  11. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  12. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  13. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  14. 2019Probabilistic Face Embeddingscited by 173
  15. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  16. 2020Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertaintycited by 101
  17. 2021MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagationcited by 82
  18. 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 69
  19. 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 67
  20. 2022Hyperbolic Contrastive Learning for Visual Representations beyond Objectscited by 61
  21. 2018Modeling Uncertainty with Hedged Instance Embeddingcited by 52
  22. 2024Learning with Mixture of Prototypes for Out-of-Distribution Detectioncited by 43
  23. 2021von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learningcited by 42
  24. 2022Uncertainty-Aware Deep Multi-View Photometric Stereocited by 30
  25. 2023Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputscited by 29
  26. 2024ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentationcited by 25
  27. 2020Enriching ImageNet with Human Similarity Judgments and Psychological Embeddingscited by 23
  28. 2022A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learningcited by 19
  29. 2022Back to the Basics: Revisiting Out-of-Distribution Detection Baselinescited by 16
  30. 2024DINO as a von Mises-Fisher mixture modelcited by 15
  31. 2020An Algorithm for Sensor Data Uncertainty Quantificationcited by 2

Built on by

Not cited yet.