# Learning from Multiple Annotator Noisy Labels via Sample-wise Label Fusion

arXiv: https://arxiv.org/abs/2207.11327 · HTML: https://mlsanity.com/paper/2207.11327/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 22 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey](https://mlsanity.com/paper/1912.05170/) (2019)
- [Who Said What: Modeling Individual Labelers Improves Classification](https://mlsanity.com/paper/1703.08774/) (2017)
- [On Symmetric Losses for Learning from Corrupted Labels](https://mlsanity.com/paper/1901.09314/) (2019)
- [Learning From Noisy Singly-labeled Data](https://mlsanity.com/paper/1712.04577/) (2017)
- [Multimodal Knowledge Expansion](https://mlsanity.com/paper/2103.14431/) (2021)
- [Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization](https://mlsanity.com/paper/2204.02485/) (2022)

## Built on by (most cited first, 10 of 10)

- [AQuA: A Benchmarking Tool for Label Quality Assessment](https://mlsanity.com/paper/2306.09467/) (2023)
- [Beyond Consensus: Perspectivist Modeling and Evaluation of Annotator Disagreement in NLP](https://mlsanity.com/paper/2601.09065/) (2026)
- [Realistic Evaluation of Deep Partial-Label Learning Algorithms](https://mlsanity.com/paper/2502.10184/) (2025)
- [Transferring Annotator- and Instance-dependent Transition Matrix for Learning from Crowds](https://mlsanity.com/paper/2306.03116/) (2023)
- [Multi-annotator Deep Learning: A Probabilistic Framework for Classification](https://mlsanity.com/paper/2304.02539/) (2023)
- [Learning to Complement with Multiple Humans](https://mlsanity.com/paper/2311.13172/) (2023)
- [Label Smarter, Not Harder: CleverLabel for Faster Annotation of Ambiguous Image Classification with Higher Quality](https://mlsanity.com/paper/2305.12811/) (2023)
- [Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels](https://mlsanity.com/paper/2309.09742/) (2023)
- [Handwritten Text Recognition from Crowdsourced Annotations](https://mlsanity.com/paper/2306.10878/) (2023)
- [Mixture of Experts based Multi-task Supervise Learning from Crowds](https://mlsanity.com/paper/2407.13268/) (2024)

## Abstract (arXiv)

Data lies at the core of modern deep learning. The impressive performance of supervised learning is built upon a base of massive accurately labeled data. However, in some real-world applications, accurate labeling might not be viable; instead, multiple noisy labels (instead of one accurate label) are provided by several annotators for each data sample. Learning a classifier on such a noisy training dataset is a challenging task. Previous approaches usually assume that all data samples share the same set of parameters related to annotator errors, while we demonstrate that label error learning should be both annotator and data sample dependent. Motivated by this observation, we propose a novel learning algorithm. The proposed method displays superiority compared with several state-of-the-art baseline methods on MNIST, CIFAR-100, and ImageNet-100. Our code is available at: https://github.com/zhengqigao/Learning-from-Multiple-Annotator-Noisy-Labels.