mlsanityData to 7 Oct 2026

Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications

arXiv 2305.06522 ↗First posted 11 May 2023cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Large-scale pre-trained language models have shown outstanding performance in a variety of NLP tasks. However, they are also known to be significantly brittle against specifically crafted adversarial examples, leading to increasing interest in probing the adversarial robustness of NLP systems. We introduce RSMI, a novel two-stage framework that combines randomized smoothing (RS) with masked inference (MI) to improve the adversarial robustness of NLP systems. RS transforms a classifier into a smoothed classifier to obtain robust representations, whereas MI forces a model to exploit the surrounding context of a masked token in an input sequence. RSMI improves adversarial robustness by 2 to 3 times over existing state-of-the-art methods on benchmark datasets. We also perform in-depth qualitative analysis to validate the effectiveness of the different stages of RSMI and probe the impact of its components through extensive ablations. By empirically proving the stability of RSMI, we put it forward as a practical method to robustly train large-scale NLP models. Our code and datasets are available at https://github.com/Han8931/rsmi_nlp

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

31 of 50 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  9. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  10. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  11. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  12. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
19 more
  1. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  2. 2018Generating Natural Language Adversarial Examplescited by 422
  3. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  4. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  5. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  6. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  7. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  8. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  9. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 158
  10. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 138
  11. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  12. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  13. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 72
  14. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  15. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 55
  16. 2020MASKER: Masked Keyword Regularization for Reliable Text Classificationcited by 25
  17. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  18. 2020Defense of Word-level Adversarial Attacks via Random Substitution Encodingcited by 9
  19. 2022How Sampling Impacts the Robustness of Stochastic Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 3; 2026: 1.

11 papers.

  1. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  2. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  3. 2025Smoothed Embeddings for Robust Language Modelscited by 8
  4. 2025Towards the Worst-case Robustness of Large Language Modelscited by 7
  5. 2024ROIC-DM: Robust Text Inference and Classification via Diffusion Modelcited by 5
  6. 2025Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removalcited by 2
  7. 2023Fooling the Textual Fooler via Randomizing Latent Representationscited by 1
  8. 2024DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoisingcited by 1
  9. 2024GenFighter: A Generative and Evolutive Textual Attack Removalcited by 0
  10. 2024Adversarial Robustness through Dynamic Ensemble Learningcited by 0
  11. 2026Directional Embedding Smoothing for Robust Vision Language Modelscited by 0