mlsanityData to 7 Oct 2026

Revisiting Character-level Adversarial Attacks for Language Models

arXiv 2405.04346 ↗First posted 7 May 2024cs.LG cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adversarial examples. While character-level attacks easily maintain semantics, they have received less attention as they cannot easily adopt popular gradient-based methods, and are thought to be easy to defend. Challenging these beliefs, we introduce Charmer, an efficient query-based adversarial attack capable of achieving high attack success rate (ASR) while generating highly similar adversarial examples. Our method successfully targets both small (BERT) and large (Llama 2) models. Specifically, on BERT with SST-2, Charmer improves the ASR in 4.84% points and the USE similarity in 8% points with respect to the previous art. Our implementation is available in https://github.com/LIONS-EPFL/Charmer.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 70 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

46 of 70 references.

References · 46

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
34 more
  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  11. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  12. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  13. 2018Generating Natural Language Adversarial Examplescited by 422
  14. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  15. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  16. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  17. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  18. 2023Are aligned neural networks adversarially aligned?cited by 305
  19. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  20. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  21. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  22. 2018Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examplescited by 93
  23. 2020Fast WordPiece Tokenizationcited by 85
  24. 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
  25. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 78
  26. 2020Robust Encodings: A Framework for Combating Adversarial Typoscited by 58
  27. 2018Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Datacited by 37
  28. 2018Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classificationcited by 34
  29. 2022Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitutioncited by 28
  30. 2022Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimizationcited by 22
  31. 2022TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimizationcited by 21
  32. 2023How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attackscited by 18
  33. 2023TransFool: An Adversarial Attack against Neural Machine Translation Modelscited by 12
  34. 2023Targeted Adversarial Attacks against Neural Machine Translationcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  2. 2025Towards the Worst-case Robustness of Large Language Modelscited by 7
  3. 2025Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarkscited by 2
  4. 2024Adversarial Robustness through Dynamic Ensemble Learningcited by 0