# Towards Robustness to Label Noise in Text Classification via Noise Modeling

arXiv: https://arxiv.org/abs/2101.11214 · HTML: https://mlsanity.com/paper/2101.11214/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Combating Label Noise in Deep Learning Using Abstention](https://mlsanity.com/paper/1905.10964/) (2019)
- [The Devil of Face Recognition is in the Noise](https://mlsanity.com/paper/1807.11649/) (2018)
- [Outlier Detection for Improved Data Quality and Diversity in Dialog Systems](https://mlsanity.com/paper/1904.03122/) (2019)
- [An Effective Label Noise Model for DNN Text Classification](https://mlsanity.com/paper/1903.07507/) (2019)
- [Hyperspectral Image Classification in the Presence of Noisy Labels](https://mlsanity.com/paper/1809.04212/) (2018)

## Built on by (most cited first, 9 of 9)

- [Exploring the Robustness of In-Context Learning with Noisy Labels](https://mlsanity.com/paper/2404.18191/) (2024)
- [Robustness of Large Language Models to Perturbations in Text](https://mlsanity.com/paper/2407.08989/) (2024)
- [SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training](https://mlsanity.com/paper/2210.04525/) (2022)
- [Robust Product Classification with Instance-Dependent Noise](https://mlsanity.com/paper/2209.06946/) (2022)
- [Combining Denoising Autoencoders with Contrastive Learning to fine-tune Transformer Models](https://mlsanity.com/paper/2405.14437/) (2024)
- [Granular-ball Representation Learning for Deep CNN on Learning with Label Noise](https://mlsanity.com/paper/2409.03254/) (2024)
- [Knowledge Distillation with Noisy Labels for Natural Language Understanding](https://mlsanity.com/paper/2109.10147/) (2021)
- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [Learning to Retrieve with Weakened Labels: Robust Training under Label Noise](https://mlsanity.com/paper/2512.13237/) (2025)

## Abstract (arXiv)

Large datasets in NLP suffer from noisy labels, due to erroneous automatic and human annotation procedures. We study the problem of text classification with label noise, and aim to capture this noise through an auxiliary noise model over the classifier. We first assign a probability score to each training sample of having a noisy label, through a beta mixture model fitted on the losses at an early epoch of training. Then, we use this score to selectively guide the learning of the noise model and classifier. Our empirical evaluation on two text classification tasks shows that our approach can improve over the baseline accuracy, and prevent over-fitting to the noise.