# A Simple Framework for Uncertainty in Contrastive Learning

arXiv: https://arxiv.org/abs/2010.02038 · HTML: https://mlsanity.com/paper/2010.02038/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [Active Learning Helps Pretrained Models Learn the Intended Task](https://mlsanity.com/paper/2204.08491/) (2022)
- [Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs](https://mlsanity.com/paper/2302.02865/) (2023)
- [Random-Set Neural Networks (RS-NN)](https://mlsanity.com/paper/2307.05772/) (2023)
- [Latent Discriminant deterministic Uncertainty](https://mlsanity.com/paper/2207.10130/) (2022)
- [Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks](https://mlsanity.com/paper/2311.01038/) (2023)
- [On Contrastive Representations of Stochastic Processes](https://mlsanity.com/paper/2106.10052/) (2021)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [Uncertainty in Contrastive Learning: On the Predictability of Downstream Performance](https://mlsanity.com/paper/2207.09336/) (2022)
- [Training, Architecture, and Prior for Deterministic Uncertainty Methods](https://mlsanity.com/paper/2303.05796/) (2023)
- [Training-Free Uncertainty Estimation for Embedding Models](https://mlsanity.com/paper/2306.00206/) (2023)
- [Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS](https://mlsanity.com/paper/2211.01999/) (2022)
- [Discriminant Distance-Aware Representation on Deterministic Uncertainty Quantification Methods](https://mlsanity.com/paper/2402.12664/) (2024)
- [Uncertainty in Graph Contrastive Learning with Bayesian Neural Networks](https://mlsanity.com/paper/2312.00232/) (2023)
- [FISHing in Uncertainty: Synthetic Contrastive Learning for Genetic Aberration Detection](https://mlsanity.com/paper/2411.01025/) (2024)
- [Contrastive Conformal Sets](https://mlsanity.com/paper/2603.26261/) (2026)

## Abstract (arXiv)

Contrastive approaches to representation learning have recently shown great promise. In contrast to generative approaches, these contrastive models learn a deterministic encoder with no notion of uncertainty or confidence. In this paper, we introduce a simple approach based on "contrasting distributions" that learns to assign uncertainty for pretrained contrastive representations. In particular, we train a deep network from a representation to a distribution in representation space, whose variance can be used as a measure of confidence. In our experiments, we show that this deep uncertainty model can be used (1) to visually interpret model behavior, (2) to detect new noise in the input to deployed models, (3) to detect anomalies, where we outperform 10 baseline methods across 11 tasks with improvements of up to 14% absolute, and (4) to classify out-of-distribution examples where our fully unsupervised model is competitive with supervised methods.