mlsanityData to 14 Oct 2026

In and Out-of-Domain Text Adversarial Robustness via Label Smoothing

arXiv 2212.10258 ↗First posted 20 Dec 2022cs.CL cs.CR cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Recently it has been shown that state-of-the-art NLP models are vulnerable to adversarial attacks, where the predictions of a model can be drastically altered by slight modifications to the input (such as synonym substitutions). While several defense techniques have been proposed, and adapted, to the discrete nature of text adversarial attacks, the benefits of general-purpose regularization methods such as label smoothing for language models, have not been studied. In this paper, we study the adversarial robustness provided by various label smoothing strategies in foundational models for diverse NLP tasks in both in-domain and out-of-domain settings. Our experiments show that label smoothing significantly improves adversarial robustness in pre-trained models like BERT, against various popular attacks. We also analyze the relationship between prediction confidence and robustness, showing that label smoothing reduces over-confident errors on adversarial examples.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 30 of its 50 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

30 of 50 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2017On Calibration of Modern Neural Networkscited by 4,960
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,847
18 more
  1. 2019When Does Label Smoothing Help?cited by 988
  2. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  3. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  4. 2018Generating Natural Language Adversarial Examplescited by 424
  5. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  6. 2020Calibration of Pre-trained Transformerscited by 315
  7. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  8. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  9. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  10. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  11. 2020Delving Deep into Label Smoothingcited by 94
  12. 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
  13. 2022SemAttack: Natural Textual Attacks via Different Semantic Spacescited by 42
  14. 2020Improving Calibration through the Relationship with Adversarial Robustnesscited by 26
  15. 2022Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimizationcited by 22
  16. 2019Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?cited by 18
  17. 2020Label Smoothing and Adversarial Robustnesscited by 10
  18. 2019Adversarial Robustness via Label-Smoothingcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 3.

5 papers.

  1. 2024Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learningcited by 4
  2. 2025Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarkscited by 2
  3. 2025GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Modelscited by 2
  4. 2025Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removalcited by 2
  5. 2023Revisiting the Role of Label Smoothing in Enhanced Text Sentiment Classificationcited by 0