mlsanityData to 14 Oct 2026

Toward Stronger Textual Attack Detectors

arXiv 2310.14001 ↗First posted 21 Oct 2023cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The landscape of available textual adversarial attacks keeps growing, posing severe threats and raising concerns regarding the deep NLP system's integrity. However, the crucial problem of defending against malicious attacks has only drawn the attention of the NLP community. The latter is nonetheless instrumental in developing robust and trustworthy systems. This paper makes two important contributions in this line of search: (i) we introduce LAROUSSE, a new framework to detect textual adversarial attacks and (ii) we introduce STAKEOUT, a new benchmark composed of nine popular attack methods, three datasets, and two pre-trained models. LAROUSSE is ready-to-use in production as it is unsupervised, hyperparameter-free, and non-differentiable, protecting it against gradient-based methods. Our new benchmark STAKEOUT allows for a robust evaluation framework: we conduct extensive numerical experiments which demonstrate that LAROUSSE outperforms previous methods, and which allows to identify interesting factors of detection rate variations.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 60 of its 104 references that are in the map. First posted on arXiv 21 Oct 2023.

Builds on

60 of 104 references.

References · 60

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  9. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  11. 2018Computational Optimal Transportcited by 780
  12. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
38 more
  1. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  2. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  3. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  4. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  5. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  6. 2018Generating Natural Language Adversarial Examplescited by 424
  7. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  8. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  9. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  10. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 286
  11. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
  12. 2017Detecting Adversarial Samples from Artifactscited by 245
  13. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 207
  14. 2019Certified Robustness to Adversarial Word Substitutionscited by 168
  15. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 163
  16. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  17. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  18. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  19. 2020Reevaluating Adversarial Examples in Natural Languagecited by 85
  20. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84
  21. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  22. 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 70
  23. 2021Towards Robustness Against Natural Language Word Substitutionscited by 69
  24. 2020Robust Encodings: A Framework for Combating Adversarial Typoscited by 58
  25. 2020Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examplescited by 54
  26. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 52
  27. 2019Affect-Driven Dialog Generationcited by 50
  28. 2021Automatic Text Evaluation through the Lens of Wasserstein Barycenterscited by 38
  29. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37
  30. 2020Hierarchical Pre-training for Sequence Labelling in Spoken Dialogcited by 36
  31. 2022Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimationcited by 32
  32. 2020Why is the Mahalanobis Distance Effective for Anomaly Detection?cited by 28
  33. 2022Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Datacited by 21
  34. 2022A Differential Entropy Estimator for Training Neural Networkscited by 20
  35. 2022Igeood: An Information Geometry Approach to Out-of-Distribution Detectioncited by 20
  36. 2021A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regionscited by 17
  37. 2020A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attackscited by 15
  38. 2023Unsupervised Layer-wise Score Aggregation for Textual OOD Detectioncited by 15

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024What If the Input is Expanded in OOD Detection?cited by 4
  2. 2024Mitigating Text Toxicity with Counterfactual Generationcited by 2
  3. 2024Signature Isolation Forestcited by 1