# NGC: A Unified Framework for Learning with Open-World Noisy Data

arXiv: https://arxiv.org/abs/2108.11035 · HTML: https://mlsanity.com/paper/2108.11035/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 58 references that are in the map.
- First posted on arXiv 25 Aug 2021.

## Builds on (most cited first, 30 of 48)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Identifying Mislabeled Data using the Area Under the Margin Ranking](https://mlsanity.com/paper/2001.10528/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)

## Built on by (most cited first, 30 of 58)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [Selective-Supervised Contrastive Learning with Noisy Labels](https://mlsanity.com/paper/2203.04181/) (2022)
- [RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection](https://mlsanity.com/paper/2209.01814/) (2022)
- [GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection](https://mlsanity.com/paper/2211.04208/) (2022)
- [OpenCon: Open-world Contrastive Learning](https://mlsanity.com/paper/2208.02764/) (2022)
- [Grow and Merge: A Unified Framework for Continuous Categories Discovery](https://mlsanity.com/paper/2210.04174/) (2022)
- [Neighborhood Collective Estimation for Noisy Label Identification and Correction](https://mlsanity.com/paper/2208.03207/) (2022)
- [PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning](https://mlsanity.com/paper/2201.08984/) (2022)
- [A CLIP-Powered Framework for Robust and Generalizable Data Selection](https://mlsanity.com/paper/2410.11215/) (2024)
- [Neural Relation Graph: A Unified Framework for Identifying Label Noise and Outlier Data](https://mlsanity.com/paper/2301.12321/) (2023)
- [Late Stopping: Avoiding Confidently Learning from Mislabeled Examples](https://mlsanity.com/paper/2308.13862/) (2023)
- [CSOT: Curriculum and Structure-Aware Optimal Transport for Learning with Noisy Labels](https://mlsanity.com/paper/2312.06221/) (2023)
- [The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition](https://mlsanity.com/paper/2302.04002/) (2023)
- [Vision-Language Models are Strong Noisy Label Detectors](https://mlsanity.com/paper/2409.19696/) (2024)
- [SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise](https://mlsanity.com/paper/2111.11288/) (2021)
- [Prototypical Classifier for Robust Class-Imbalanced Learning](https://mlsanity.com/paper/2110.11553/) (2021)
- [L2B: Learning to Bootstrap Robust Models for Combating Label Noise](https://mlsanity.com/paper/2202.04291/) (2022)
- [BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classification](https://mlsanity.com/paper/2203.01937/) (2022)
- [Sample Prior Guided Robust Model Learning to Suppress Noisy Labels](https://mlsanity.com/paper/2112.01197/) (2021)
- [Unlocking the Power of Open Set : A New Perspective for Open-Set Noisy Label Learning](https://mlsanity.com/paper/2305.04203/) (2023)
- [Noisy Label Processing for Classification: A Survey](https://mlsanity.com/paper/2404.04159/) (2024)
- [Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future](https://mlsanity.com/paper/2410.05577/) (2024)
- [Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels](https://mlsanity.com/paper/2210.08826/) (2022)
- [Rethinking Out-of-Distribution Detection From a Human-Centric Perspective](https://mlsanity.com/paper/2211.16778/) (2022)
- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [Combating Label Noise With A General Surrogate Model For Sample Selection](https://mlsanity.com/paper/2310.10463/) (2023)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [Learning with Noisy Labels over Imbalanced Subpopulations](https://mlsanity.com/paper/2211.08722/) (2022)

## Abstract (arXiv)

The existence of noisy data is prevalent in both the training and testing phases of machine learning systems, which inevitably leads to the degradation of model performance. There have been plenty of works concentrated on learning with in-distribution (IND) noisy labels in the last decade, i.e., some training samples are assigned incorrect labels that do not correspond to their true classes. Nonetheless, in real application scenarios, it is necessary to consider the influence of out-of-distribution (OOD) samples, i.e., samples that do not belong to any known classes, which has not been sufficiently explored yet. To remedy this, we study a new problem setup, namely Learning with Open-world Noisy Data (LOND). The goal of LOND is to simultaneously learn a classifier and an OOD detector from datasets with mixed IND and OOD noise. In this paper, we propose a new graph-based framework, namely Noisy Graph Cleaning (NGC), which collects clean samples by leveraging geometric structure of data and model predictive confidence. Without any additional training effort, NGC can detect and reject the OOD samples based on the learned class prototypes directly in testing phase. We conduct experiments on multiple benchmarks with different types of noise and the results demonstrate the superior performance of our method against state of the arts.