# Holistic Uncertainty Estimation For Open-Set Recognition

arXiv: https://arxiv.org/abs/2408.14229 · HTML: https://mlsanity.com/paper/2408.14229/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 17 of 17)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [Nonparametric Uncertainty Quantification for Single Deterministic Neural Network](https://mlsanity.com/paper/2202.03101/) (2022)
- [Toward Open-Set Face Recognition](https://mlsanity.com/paper/1705.01567/) (2017)
- [VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark](https://mlsanity.com/paper/2407.11510/) (2024)
- [Stating Comparison Score Uncertainty and Verification Decision Confidence Towards Transparent Face Recognition](https://mlsanity.com/paper/2210.10354/) (2022)
- [Probabilistic Embeddings Revisited](https://mlsanity.com/paper/2202.06768/) (2022)
- [Harnessing Unrecognizable Faces for Improving Face Recognition](https://mlsanity.com/paper/2106.04112/) (2021)
- [ScaleFace: Uncertainty-aware Deep Metric Learning](https://mlsanity.com/paper/2209.01880/) (2022)

## Built on by (most cited first, 3 of 3)

- [Conformal Inference for Open-Set and Imbalanced Classification](https://mlsanity.com/paper/2510.13037/) (2025)
- [Uncertainty Estimation for the Open-Set Text Classification systems](https://mlsanity.com/paper/2604.08560/) (2026)
- [Mixed-Prior Decision Risk for Open-Set Recognition](https://mlsanity.com/paper/2609.35043/) (2026)

## Abstract (arXiv)

Accurate uncertainty estimation is a critical challenge in open-set recognition, where a probe biometric sample may belong to an unknown identity. It can be addressed through sample quality estimation via probabilistic embeddings. However, the low variance of probabilistic embedding only partly implies a low identification error probability: an embedding of a sample could be close to several classes in a gallery, thus yielding high uncertainty despite high sample quality. We propose HolUE - a holistic uncertainty estimation method based on a Bayesian probabilistic model; it is aware of two sources of ambiguity in the open-set recognition system: (1) the gallery uncertainty caused by overlapping classes and (2) the uncertainty of embeddings. Challenging open-set recognition datasets, such as IJB-C for the image domain and VoxBlink for the audio domain, serve as a testbed for our method. We also provide a new open-set recognition protocol for the identification of whales and dolphins. In all cases, HolUE better identifies recognition errors than alternative uncertainty estimation methods, including those based solely on sample quality.