# Understanding Model Robustness to User-generated Noisy Texts

arXiv: https://arxiv.org/abs/2110.07428 · HTML: https://mlsanity.com/paper/2110.07428/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://mlsanity.com/paper/1805.02266/) (2018)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [MTNT: A Testbed for Machine Translation of Noisy Text](https://mlsanity.com/paper/1809.00388/) (2018)
- [Grammatical Error Correction in Low-Resource Scenarios](https://mlsanity.com/paper/1910.00353/) (2019)
- [A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning](https://mlsanity.com/paper/1907.01256/) (2019)
- [How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?](https://mlsanity.com/paper/1704.04441/) (2017)
- [Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study](https://mlsanity.com/paper/1807.01270/) (2018)
- [Findings of the First Shared Task on Machine Translation Robustness](https://mlsanity.com/paper/1906.11943/) (2019)
- [Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords](https://mlsanity.com/paper/2007.03006/) (2020)
- [Artificial Error Generation with Machine Translation and Syntactic Patterns](https://mlsanity.com/paper/1707.05236/) (2017)
- [Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection](https://mlsanity.com/paper/1810.00668/) (2018)
- [Evaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing](https://mlsanity.com/paper/1908.07448/) (2019)
- [Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer](https://mlsanity.com/paper/2007.01667/) (2020)

## Built on by (most cited first, 8 of 8)

- [Czech Grammar Error Correction with a Large and Diverse Corpus](https://mlsanity.com/paper/2201.05590/) (2022)
- [Robustness of Large Language Models to Perturbations in Text](https://mlsanity.com/paper/2407.08989/) (2024)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [Scrambled text: training Language Models to correct OCR errors using synthetic data](https://mlsanity.com/paper/2409.19735/) (2024)
- [READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises](https://mlsanity.com/paper/2302.07324/) (2023)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages](https://mlsanity.com/paper/2601.16623/) (2026)
- [An Understanding-Oriented Robust Machine Reading Comprehension Model](https://mlsanity.com/paper/2207.00187/) (2022)

## Abstract (arXiv)

Sensitivity of deep-neural models to input noise is known to be a challenging problem. In NLP, model performance often deteriorates with naturally occurring noise, such as spelling errors. To mitigate this issue, models may leverage artificially noised data. However, the amount and type of generated noise has so far been determined arbitrarily. We therefore propose to model the errors statistically from grammatical-error-correction corpora. We present a thorough evaluation of several state-of-the-art NLP systems' robustness in multiple languages, with tasks including morpho-syntactic analysis, named entity recognition, neural machine translation, a subset of the GLUE benchmark and reading comprehension. We also compare two approaches to address the performance drop: a) training the NLP models with noised data generated by our framework; and b) reducing the input noise with external system for natural language correction. The code is released at https://github.com/ufal/kazitext.