mlsanityData to 14 Oct 2026

From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attacks

arXiv 2010.05648 ↗First posted 12 Oct 2020cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

Adversarial attacks are label-preserving modifications to inputs of machine learning classifiers designed to fool machines but not humans. Natural Language Processing (NLP) has mostly focused on high-level attack scenarios such as paraphrasing input texts. We argue that these are less realistic in typical application scenarios such as in social media, and instead focus on low-level attacks on the character-level. Guided by human cognitive abilities and human robustness, we propose the first large-scale catalogue and benchmark of low-level adversarial attacks, which we dub Z'eroe, encompassing nine different attack modes including visual and phonetic adversaries. We show that RoBERTa, NLP's current workhorse, fails on our attacks. Our dataset provides a benchmark for testing robustness of future more human-like NLP models.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 6; 2024: 8; 2025: 7; 2026: 3.

36 papers.

Sort:

By area:

  1. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 210
  2. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  3. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  4. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 65
  5. 2022Language Modelling with Pixelscited by 55
  6. 2022Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitutioncited by 28
  7. 2021Reliability Testing for Natural Language Processing Systemscited by 25
  8. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
  9. 2021BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attackscited by 17
  10. 2024PIXAR: Auto-Regressive Language Modeling in Pixel Spacecited by 16
  11. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  12. 2025ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solvingcited by 13
24 more
  1. 2021Token-Modification Adversarial Attacks for Natural Language Processing: A Surveycited by 12
  2. 2022Layer or Representation Space: What makes BERT-based Evaluation Metrics Robust?cited by 10
  3. 2023Byte-Level Grammatical Error Correction Using Synthetic and Curated Corporacited by 7
  4. 2022ROSE: Robust Selective Fine-tuning for Pre-trained Language Modelscited by 6
  5. 2022The Risks of Machine Learning Systemscited by 5
  6. 2022Adversarial Text Normalizationcited by 5
  7. 2023Break it, Imitate it, Fix it: Robustness by Generating Human-Like Attackscited by 5
  8. 2024ChatGPT Incorrectness Detection in Software Reviewscited by 5
  9. 2022Exposing Fine-Grained Adversarial Vulnerability of Face Anti-Spoofing Modelscited by 3
  10. 2023Learning the Legibility of Visual Text Perturbationscited by 3
  11. 2023From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Frameworkcited by 3
  12. 2024SemRoDe: Macro Adversarial Training to Learn Representations That are Robust to Word-Level Attackscited by 3
  13. 2025FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluationcited by 3
  14. 2024CERT-ED: Certifiably Robust Text Classification for Edit Distancecited by 2
  15. 2025Pay Attention to Real World Perturbations! Natural Robustness Evaluation in Machine Reading Comprehensioncited by 2
  16. 2025Memory Self-Regeneration: Uncovering Hidden Knowledge in Unlearned Modelscited by 2
  17. 2024Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognitioncited by 1
  18. 2024Pairing Orthographically Variant Literary Words to Standard Equivalents Using Neural Edit Distance Modelscited by 1
  19. 2025Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approachcited by 1
  20. 2025More Agents Improve Math Problem Solving but Adversarial Robustness Gap Persistscited by 1
  21. 2026MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scriptscited by 1
  22. 2025VISAT: Benchmarking Adversarial and Distribution Shift Robustness in Traffic Sign Recognition with Visual Attributescited by 0
  23. 2026S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLPcited by 0
  24. 2026Evaluation of Adversarial Robustness in Arabic Language Modelscited by 0