Data to 14 Oct 2026

Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT

arXiv 2003.04985 ↗First posted 27 Feb 2020cs.CLcited by 69

Not on the map. Why?

Abstract (arXiv)

There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously. It is unclear, however, how the models will perform in realistic scenarios where textitnatural rather than malicious adversarial instances often exist. This work systematically explores the robustness of BERT, the state-of-the-art Transformer-style model in NLP, in dealing with noisy data, particularly mistakes in typing the keyboard, that occur inadvertently. Intensive experiments on sentiment analysis and question answering benchmarks indicate that: (i) Typos in various words of a sentence do not influence equally. The typos in informative words make severer damages; (ii) Mistype is the most damaging factor, compared with inserting, deleting, etc.; (iii) Humans and machines have different focuses on recognizing adversarial attacks.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 18; 2023: 13; 2024: 7; 2025: 7; 2026: 4.

69 papers.

Sort:

By area:

2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 232
2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 160
2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
2021User Preference-aware Fake News Detectioncited by 79
2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 72
2023Black Box Adversarial Prompting for Foundation Modelscited by 62

38 more

2022Language Modelling with Pixelscited by 55
2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 53
2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 52
2021Named Entity Recognition and Classification on Historical Documents: A Surveycited by 47
2021Robust Transfer Learning with Pretrained Language Models through Adapterscited by 35
2022CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typoscited by 22
2023Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Valuescited by 21
2022How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?cited by 20
2021The King is Naked: on the Notion of Robustness for Natural Language Processingcited by 16
2024Retrofitting Large Language Models with Dynamic Tokenizationcited by 16
2022Fooling Explanations in Text Classifierscited by 13
2021Transformers: "The End of History" for NLP?cited by 12
2021Contrastive Fine-tuning Improves Robustness for Neural Rankerscited by 12
2021Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?cited by 11
2022Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Costcited by 9
2025Evaluating Robustness of Large Language Models Against Multilingual Typographical Errorscited by 9
2022Short-answer scoring with ensembles of pretrained language modelscited by 8
2023On Robustness of Finetuned Transformer-based NLP Modelscited by 8
2021Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviewscited by 7
2022ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attentioncited by 6
2022HashFormers: Towards Vocabulary-independent Pre-trained Transformerscited by 6
2022Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answeringcited by 6
2022An Assessment of the Impact of OCR Noise on Language Modelscited by 5
2022Local Byte Fusion for Neural Machine Translationcited by 5
2024A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Modelscited by 5
2022Visualizing and Explaining Language Modelscited by 4
2023Benchmarking and Analyzing Robust Point Cloud Recognition: Bag of Tricks for Defending Adversarial Examplescited by 4
2023To token or not to token: A Comparative Study of Text Representations for Cross-Lingual Transfercited by 4
2021Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Miningcited by 3
2021Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attackcited by 3
2021Noisy Text Data: Achilles' Heel of popular transformer based NLP modelscited by 3
2022Rare but Severe Neural Machine Translation Errors Induced by Minimal Deletion: An Empirical Study on Chinese and Englishcited by 3
2026Proxy Compression for Language Modelingcited by 3
2026Fast Byte Latent Transformercited by 3
2021Adversarial Examples for Extreme Multilabel Text Classificationcited by 2
2022Exploring Robustness of Prefix Tuning in Noisy Data: A Case Study in Financial Sentiment Analysiscited by 2
2023LEA: Improving Sentence Similarity Robustness to Typos Using Lexical Attention Biascited by 2
2024Can AI Models Appreciate Document Aesthetics? An Exploration of Legibility and Layout Quality in Relation to Prediction Confidencecited by 2

50 most cited of 69 All 69 (JSON)