# EvidentialMix: Learning with Combined Open-set and Closed-set Noisy Labels

arXiv: https://arxiv.org/abs/2011.05704 · HTML: https://mlsanity.com/paper/2011.05704/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 28 of 28)

- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [NGC: A Unified Framework for Learning with Open-World Noisy Data](https://mlsanity.com/paper/2108.11035/) (2021)
- [LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment](https://mlsanity.com/paper/2103.04173/) (2021)
- [SimT: Handling Open-set Noise for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.15202/) (2022)
- [Addressing out-of-distribution label noise in webly-labelled data](https://mlsanity.com/paper/2110.13699/) (2021)
- [ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learning](https://mlsanity.com/paper/2103.11395/) (2021)
- [SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise](https://mlsanity.com/paper/2111.11288/) (2021)
- [A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?](https://mlsanity.com/paper/2404.01775/) (2024)
- [Is your noise correction noisy? PLS: Robustness to label noise with two stage detection](https://mlsanity.com/paper/2210.04578/) (2022)
- [Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names](https://mlsanity.com/paper/2408.00298/) (2024)
- [Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise](https://mlsanity.com/paper/2308.06861/) (2023)
- [A Survey on Open-Set Image Recognition](https://mlsanity.com/paper/2312.15571/) (2023)
- [Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2204.09120/) (2022)
- [Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets](https://mlsanity.com/paper/2207.01573/) (2022)
- [ANNE: Adaptive Nearest Neighbors and Eigenvector-based Sample Selection for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2411.01613/) (2024)
- [Tackling Instance-Dependent Label Noise with Dynamic Distribution Calibration](https://mlsanity.com/paper/2210.05126/) (2022)
- [Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective](https://mlsanity.com/paper/2307.16889/) (2023)
- [Friends and Foes in Learning from Noisy Labels](https://mlsanity.com/paper/2103.15055/) (2021)
- [PASS: Peer-Agreement based Sample Selection for training with Noisy Labels](https://mlsanity.com/paper/2303.10802/) (2023)
- [Label Noise: Correcting the Forward-Correction](https://mlsanity.com/paper/2307.13100/) (2023)
- [Unleashing the Potential of Open-set Noisy Samples Against Label Noise for Medical Image Classification](https://mlsanity.com/paper/2406.12293/) (2024)
- [Noisy Early Stopping for Noisy Labels](https://mlsanity.com/paper/2409.06830/) (2024)
- [Open set label noise learning with robust sample selection and margin-guided module](https://mlsanity.com/paper/2501.04269/) (2025)
- [Dropout can Simulate Exponential Number of Models for Sample Selection Techniques](https://mlsanity.com/paper/2202.13203/) (2022)
- [Partial-label Learning with Mixed Closed-set and Open-set Out-of-candidate Examples](https://mlsanity.com/paper/2307.00553/) (2023)
- [Soft Curriculum for Learning Conditional GANs with Noisy-Labeled and Uncurated Unlabeled Data](https://mlsanity.com/paper/2307.08319/) (2023)
- [An accurate detection is not all you need to combat label noise in web-noisy datasets](https://mlsanity.com/paper/2407.05528/) (2024)
- [Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction](https://mlsanity.com/paper/2608.30046/) (2026)

## Abstract (arXiv)

The efficacy of deep learning depends on large-scale data sets that have been carefully curated with reliable data acquisition and annotation processes. However, acquiring such large-scale data sets with precise annotations is very expensive and time-consuming, and the cheap alternatives often yield data sets that have noisy labels. The field has addressed this problem by focusing on training models under two types of label noise: 1) closed-set noise, where some training samples are incorrectly annotated to a training label other than their known true class; and 2) open-set noise, where the training set includes samples that possess a true class that is (strictly) not contained in the set of known training labels. In this work, we study a new variant of the noisy label problem that combines the open-set and closed-set noisy labels, and introduce a benchmark evaluation to assess the performance of training algorithms under this setup. We argue that such problem is more general and better reflects the noisy label scenarios in practice. Furthermore, we propose a novel algorithm, called EvidentialMix, that addresses this problem and compare its performance with the state-of-the-art methods for both closed-set and open-set noise on the proposed benchmark. Our results show that our method produces superior classification results and better feature representations than previous state-of-the-art methods. The code is available at https://github.com/ragavsachdeva/EvidentialMix.