# An Effective Label Noise Model for DNN Text Classification

arXiv: https://arxiv.org/abs/1903.07507 · HTML: https://mlsanity.com/paper/1903.07507/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2019.

## Built on by (most cited first, 22 of 22)

- [Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction](https://mlsanity.com/paper/2102.09427/) (2021)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification](https://mlsanity.com/paper/2204.09371/) (2022)
- [BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning](https://mlsanity.com/paper/2211.05610/) (2022)
- [Exploring the Robustness of In-Context Learning with Noisy Labels](https://mlsanity.com/paper/2404.18191/) (2024)
- [Evaluating Multi-label Classifiers with Noisy Labels](https://mlsanity.com/paper/2102.08427/) (2021)
- [Clean or Annotate: How to Spend a Limited Data Collection Budget](https://mlsanity.com/paper/2110.08355/) (2021)
- [Towards Robustness to Label Noise in Text Classification via Noise Modeling](https://mlsanity.com/paper/2101.11214/) (2021)
- [CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise](https://mlsanity.com/paper/2105.07338/) (2021)
- [SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training](https://mlsanity.com/paper/2210.04525/) (2022)
- [CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning](https://mlsanity.com/paper/2411.00873/) (2024)
- [Handling Realistic Label Noise in BERT Text Classification](https://mlsanity.com/paper/2305.16337/) (2023)
- [Training Dynamic based data filtering may not work for NLP datasets](https://mlsanity.com/paper/2109.09191/) (2021)
- [Robust Product Classification with Instance-Dependent Noise](https://mlsanity.com/paper/2209.06946/) (2022)
- [Typing Errors in Factual Knowledge Graphs: Severity and Possible Ways Out](https://mlsanity.com/paper/2102.02307/) (2021)
- [AlleNoise: large-scale text classification benchmark dataset with real-world label noise](https://mlsanity.com/paper/2407.10992/) (2024)
- [Granular-ball Representation Learning for Deep CNN on Learning with Label Noise](https://mlsanity.com/paper/2409.03254/) (2024)
- [Knowledge Distillation with Noisy Labels for Natural Language Understanding](https://mlsanity.com/paper/2109.10147/) (2021)
- [Task-Adaptive Pre-Training for Boosting Learning With Noisy Labels: A Study on Text Classification for African Languages](https://mlsanity.com/paper/2206.01476/) (2022)
- [Context-based Virtual Adversarial Training for Text Classification with Noisy Labels](https://mlsanity.com/paper/2206.11851/) (2022)
- [NoisyAG-News: A Benchmark for Addressing Instance-Dependent Noise in Text Classification](https://mlsanity.com/paper/2407.06579/) (2024)
- [CoV-TI-Net: Transferred Initialization with Modified End Layer for COVID-19 Diagnosis](https://mlsanity.com/paper/2209.09556/) (2022)

## Abstract (arXiv)

Because large, human-annotated datasets suffer from labeling errors, it is crucial to be able to train deep neural networks in the presence of label noise. While training image classification models with label noise have received much attention, training text classification models have not. In this paper, we propose an approach to training deep networks that is robust to label noise. This approach introduces a non-linear processing layer (noise model) that models the statistics of the label noise into a convolutional neural network (CNN) architecture. The noise model and the CNN weights are learned jointly from noisy training data, which prevents the model from overfitting to erroneous labels. Through extensive experiments on several text classification datasets, we show that this approach enables the CNN to learn better sentence representations and is robust even to extreme label noise. We find that proper initialization and regularization of this noise model is critical. Further, by contrast to results focusing on large batch sizes for mitigating label noise for image classification, we find that altering the batch size does not have much effect on classification performance.