mlsanityData to 7 Oct 2026

Exploring Gradient-Guided Masked Language Model to Detect Textual Adversarial Attacks

arXiv 2504.08798 ↗First posted 8 Apr 2025cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Textual adversarial examples pose serious threats to the reliability of natural language processing systems. Recent studies suggest that adversarial examples tend to deviate from the underlying manifold of normal texts, whereas pre-trained masked language models can approximate the manifold of normal data. These findings inspire the exploration of masked language models for detecting textual adversarial attacks. We first introduce Masked Language Model-based Detection (MLMD), leveraging the mask and unmask operations of the masked language modeling (MLM) objective to induce the difference in manifold changes between normal and adversarial texts. Although MLMD achieves competitive detection performance, its exhaustive one-by-one masking strategy introduces significant computational overhead. Our posterior analysis reveals that a significant number of non-keywords in the input are not important for detection but consume resources. Building on this, we introduce Gradient-guided MLMD (GradMLMD), which leverages gradient information to identify and skip non-keywords during detection, significantly reducing resource consumption without compromising detection performance.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

39 of 59 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2021Membership Inference Attacks From First Principlescited by 839
  12. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
27 more
  1. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  2. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  3. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  4. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
  5. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  6. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  7. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 94
  8. 2016A Boundary Tilting Persepective on the Phenomenon of Adversarial Examplescited by 70
  9. 2021Bad Characters: Imperceptible NLP Attackscited by 69
  10. 2019Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classificationcited by 55
  11. 2020Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examplescited by 54
  12. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  13. 2019Natural Language Adversarial Defense through Synonym Encodingcited by 39
  14. 2021Unsupervised Out-of-Domain Detection via Pre-trained Transformerscited by 31
  15. 2022"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attackscited by 24
  16. 2022Textual Manifold-based Defense Against Natural Language Adversarial Examplescited by 19
  17. 2021Efficient Combinatorial Optimization for Word-level Adversarial Textual Attackcited by 17
  18. 2022Text Adversarial Purification as Defense against Adversarial Attackscited by 15
  19. 2024TTSlow: Slow Down Text-to-Speech with Efficiency Robustness Evaluationscited by 12
  20. 2023Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descendcited by 9
  21. 2023TextShield: Beyond Successfully Detecting Adversarial Sentences in Text Classificationcited by 8
  22. 2023Don't Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Textcited by 8
  23. 2021Self-Supervised Adversarial Example Detection by Disentangled Representationcited by 7
  24. 2022Evaluating Membership Inference Through Adversarial Robustnesscited by 7
  25. 2022Rethinking Textual Adversarial Defense for Pre-trained Language Modelscited by 5
  26. 2023Masked Language Model Based Textual Adversarial Example Detectioncited by 5
  27. 2022Disentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustnesscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025BiMark: Unbiased Multilayer Watermarking for Large Language Modelscited by 18
  2. 2025Character-Level Perturbations Disrupt LLM Watermarkscited by 7
  3. 2026Are You Still the Agent I Authorized? Earned Authority under a Fixed Ceiling for Evolving Agentscited by 4
  4. 2025Text Adversarial Attacks with Dynamic Outputscited by 2