# Revisiting the Role of Label Smoothing in Enhanced Text Sentiment Classification

arXiv: https://arxiv.org/abs/2312.06522 · HTML: https://mlsanity.com/paper/2312.06522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [A Survey on Text Classification: From Shallow to Deep Learning](https://mlsanity.com/paper/2008.00364/) (2020)
- [LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification](https://mlsanity.com/paper/2101.03305/) (2021)
- [Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?](https://mlsanity.com/paper/2206.14532/) (2022)
- [Regularization via Structural Label Smoothing](https://mlsanity.com/paper/2001.01900/) (2020)
- [Boundary Smoothing for Named Entity Recognition](https://mlsanity.com/paper/2204.12031/) (2022)
- [Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage](https://mlsanity.com/paper/2006.11991/) (2020)
- [Towards Speaker Age Estimation with Label Distribution Learning](https://mlsanity.com/paper/2202.11424/) (2022)
- [Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks](https://mlsanity.com/paper/2202.13840/) (2022)
- [An Investigation of how Label Smoothing Affects Generalization](https://mlsanity.com/paper/2010.12648/) (2020)
- [Naive Bayes and Text Classification I - Introduction and Theory](https://mlsanity.com/paper/1410.5329/) (2014)
- [In and Out-of-Domain Text Adversarial Robustness via Label Smoothing](https://mlsanity.com/paper/2212.10258/) (2022)
- [Improving Generalization of Deep Neural Network Acoustic Models with Length Perturbation and N-best Based Label Smoothing](https://mlsanity.com/paper/2203.15176/) (2022)
- [Label Smoothing Improves Neural Source Code Summarization](https://mlsanity.com/paper/2303.16178/) (2023)

## Abstract (arXiv)

Label smoothing is a widely used technique in various domains, such as text classification, image classification and speech recognition, known for effectively combating model overfitting. However, there is little fine-grained analysis on how label smoothing enhances text sentiment classification. To fill in the gap, this article performs a set of in-depth analyses on eight datasets for text sentiment classification and three deep learning architectures: TextCNN, BERT, and RoBERTa, under two learning schemes: training from scratch and fine-tuning. By tuning the smoothing parameters, we can achieve improved performance on almost all datasets for each model architecture. We further investigate the benefits of label smoothing, finding that label smoothing can accelerate the convergence of deep models and make samples of different labels easily distinguishable.