# What to do about non-standard (or non-canonical) language in NLP

arXiv: https://arxiv.org/abs/1608.07836 · HTML: https://mlsanity.com/paper/1608.07836/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2016.

## Built on by (most cited first, 30 of 35)

- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [NLPeer: A Unified Resource for the Computational Study of Peer Review](https://mlsanity.com/paper/2211.06651/) (2022)
- [Named Entity Recognition and Classification on Historical Documents: A Survey](https://mlsanity.com/paper/2109.11406/) (2021)
- [Efficient Hierarchical Domain Adaptation for Pretrained Language Models](https://mlsanity.com/paper/2112.08786/) (2021)
- [Reward Gaming in Conditional Text Generation](https://mlsanity.com/paper/2211.08714/) (2022)
- [Reliability Testing for Natural Language Processing Systems](https://mlsanity.com/paper/2105.02590/) (2021)
- [ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment](https://mlsanity.com/paper/2305.14463/) (2023)
- [GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation](https://mlsanity.com/paper/2306.01966/) (2023)
- [Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond](https://mlsanity.com/paper/2201.11443/) (2022)
- [Dense Retrieval Adaptation using Target Domain Description](https://mlsanity.com/paper/2307.02740/) (2023)
- [Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks](https://mlsanity.com/paper/2111.01340/) (2021)
- [Detecting Unassimilated Borrowings in Spanish: An Annotated Corpus and Approaches to Modeling](https://mlsanity.com/paper/2203.16169/) (2022)
- [Language and Speech Technology for Central Kurdish Varieties](https://mlsanity.com/paper/2403.01983/) (2024)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [How Universal is Genre in Universal Dependencies?](https://mlsanity.com/paper/2112.04971/) (2021)
- [Event Coreference Resolution for Contentious Politics Events](https://mlsanity.com/paper/2203.10123/) (2022)
- [Understanding Domain Learning in Language Models Through Subpopulation Analysis](https://mlsanity.com/paper/2210.12553/) (2022)
- [What the Weight?! A Unified Framework for Zero-Shot Knowledge Composition](https://mlsanity.com/paper/2401.12756/) (2024)
- [AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text](https://mlsanity.com/paper/2503.18247/) (2025)
- [Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression](https://mlsanity.com/paper/2407.13141/) (2024)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [Second language Korean Universal Dependency treebank v1.2: Focus on data augmentation and annotation scheme refinement](https://mlsanity.com/paper/2503.14718/) (2025)
- [Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains](https://mlsanity.com/paper/2204.04534/) (2022)
- [Can Humans Identify Domains?](https://mlsanity.com/paper/2404.01785/) (2024)
- [Dialetto, ma Quanto Dialetto? Transcribing and Evaluating Dialects on a Continuum](https://mlsanity.com/paper/2410.14589/) (2024)
- [Variation between Credible and Non-Credible News Across Topics](https://mlsanity.com/paper/2411.12458/) (2024)
- [Leveraging Lora Fine-Tuning and Knowledge Bases for Construction Identification](https://mlsanity.com/paper/2601.13105/) (2026)
- [MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages](https://mlsanity.com/paper/2601.16623/) (2026)
- [An Inclusive Notion of Text](https://mlsanity.com/paper/2211.05604/) (2022)

## Abstract (arXiv)

Real world data differs radically from the benchmark corpora we use in natural language processing (NLP). As soon as we apply our technologies to the real world, performance drops. The reason for this problem is obvious: NLP models are trained on samples from a limited set of canonical varieties that are considered standard, most prominently English newswire. However, there are many dimensions, e.g., socio-demographics, language, genre, sentence type, etc. on which texts can differ from the standard. The solution is not obvious: we cannot control for all factors, and it is not clear how to best go beyond the current practice of training on homogeneous data from a single domain and language. In this paper, I review the notion of canonicity, and how it shapes our community's approach to language. I argue for leveraging what I call fortuitous data, i.e., non-obvious data that is hitherto neglected, hidden in plain sight, or raw data that needs to be refined. If we embrace the variety of this heterogeneous data by combining it with proper algorithms, we will not only produce more robust models, but will also enable adaptive language technology capable of addressing natural language variation.