# ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2103.11395 · HTML: https://mlsanity.com/paper/2103.11395/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 66 references that are in the map.
- First posted on arXiv 21 Mar 2021.

## Builds on (most cited first, 30 of 46)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise](https://mlsanity.com/paper/1711.07131/) (2017)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)

## Built on by (most cited first, 13 of 13)

- [Is your noise correction noisy? PLS: Robustness to label noise with two stage detection](https://mlsanity.com/paper/2210.04578/) (2022)
- [Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names](https://mlsanity.com/paper/2408.00298/) (2024)
- [Dynamics-Aware Loss for Learning with Label Noise](https://mlsanity.com/paper/2303.11562/) (2023)
- [Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise](https://mlsanity.com/paper/2308.06861/) (2023)
- [Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels](https://mlsanity.com/paper/2210.08826/) (2022)
- [Learning to Complement and to Defer to Multiple Users](https://mlsanity.com/paper/2407.07003/) (2024)
- [Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets](https://mlsanity.com/paper/2207.01573/) (2022)
- [PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive Representation Learning](https://mlsanity.com/paper/2402.17589/) (2024)
- [GANN: Graph Alignment Neural Network for Semi-Supervised Learning](https://mlsanity.com/paper/2303.07778/) (2023)
- [An Interpretable Power System Transient Stability Assessment Method with Expert Guiding Neural-Regression-Tree](https://mlsanity.com/paper/2404.02555/) (2024)
- [A Multi-module Robust Method for Transient Stability Assessment against False Label Injection Cyberattacks](https://mlsanity.com/paper/2406.06744/) (2024)
- [An accurate detection is not all you need to combat label noise in web-noisy datasets](https://mlsanity.com/paper/2407.05528/) (2024)
- [Prototype-Guided Pseudo-Labeling with Neighborhood-Aware Consistency for Unsupervised Adaptation](https://mlsanity.com/paper/2507.22075/) (2025)

## Abstract (arXiv)

We propose a new training algorithm, ScanMix, that explores semantic clustering and semi-supervised learning (SSL) to allow superior robustness to severe label noise and competitive robustness to non-severe label noise problems, in comparison to the state of the art (SOTA) methods. ScanMix is based on the expectation maximisation framework, where the E-step estimates the latent variable to cluster the training images based on their appearance and classification results, and the M-step optimises the SSL classification and learns effective feature representations via semantic clustering. We present a theoretical result that shows the correctness and convergence of ScanMix, and an empirical result that shows that ScanMix has SOTA results on CIFAR-10/-100 (with symmetric, asymmetric and semantic label noise), Red Mini-ImageNet (from the Controlled Noisy Web Labels), Clothing1M and WebVision. In all benchmarks with severe label noise, our results are competitive to the current SOTA.