mlsanityData to 7 Oct 2026

S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP

arXiv 2606.13439 ↗First posted 11 Jun 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite recent progress in Natural Language Processing (NLP), models remain vulnerable to word substitution attacks. Most existing defenses focus on first order sensitivity and measure how much the output changes when the input is slightly perturbed. However, they ignore how this sensitivity evolves, which is described by curvature. When gradients vary sharply, models can still fail. This paper introduces the Smooth Growth Bound Tensor (S-GBT), a second order method that bounds the Hessian element-wise, for which we provide formal theoretical proofs on the resulting robustness bounds. A regularization term is added during training to minimize these bounds. This yields tighter certified robustness against word substitution attacks. The change in the output under word substitution is bounded by both a linear term and a quadratic term. S-GBT is derived for two architectures: Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). The method is integrated directly into the training objective. Its effectiveness is evaluated on multiple benchmark datasets. The results show that combining first and second order regularization improves certified robust accuracy by up to 23.4% compared to prior methods, while clean accuracy remains competitive. These findings indicate that controlling both the gradient and its variation is a promising direction for building more robust models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

27 of 39 references.

References · 27

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  4. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  5. 2018Generating Natural Language Adversarial Examplescited by 422
  6. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  7. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  8. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  9. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  10. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  11. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  12. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84
15 more
  1. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 72
  2. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  3. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  4. 2020Universal Adversarial Attacks with Natural Triggers for Text Classificationcited by 50
  5. 2020Generating Natural Language Attacks in a Hard Label Black Box Settingcited by 50
  6. 2019Natural Language Adversarial Defense through Synonym Encodingcited by 39
  7. 2020From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attackscited by 36
  8. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 32
  9. 2020Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemblecited by 26
  10. 2021Achieving Model Robustness through Discrete Adversarial Trainingcited by 23
  11. 2024Assessing Adversarial Robustness of Large Language Models: An Empirical Studycited by 21
  12. 2021Certified Robustness to Programmable Transformations in LSTMscited by 18
  13. 2023Generating Valid and Natural Adversarial Examples with Large Language Modelscited by 12
  14. 2024Exploring Adversarial Robustness of Deep State Space Modelscited by 4
  15. 2025Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approachcited by 1

Built on by

Not cited yet.