mlsanityData to 14 Oct 2026

Don't Touch My Diacritics

arXiv 2410.24140 ↗First posted 31 Oct 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

The common practice of preprocessing text before feeding it into NLP models introduces many decision points which have unintended consequences on model performance. In this opinion piece, we focus on the handling of diacritics in texts originating in many languages and scripts. We demonstrate, through several case studies, the adverse effects of inconsistent encoding of diacritized characters and of removing diacritics altogether. We call on the community to adopt simple but necessary steps across all models and toolkits in order to improve handling of diacritized text and, by extension, increase equity in multilingual NLP.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 12 of its 24 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

12 of 24 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  5. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  6. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 232
  7. 2018Marian: Fast Neural Machine Translation in C++cited by 203
  8. 2021How to Train BERT with an Academic Budgetcited by 80
  9. 2020GREEK-BERT: The Greeks visiting Sesame Streetcited by 38
  10. 2023Text Rendering Strategies for Pixel Language Modelscited by 22
  11. 2021Restoring Hebrew Diacritics Without a Dictionarycited by 10
  12. 2022Analyzing Gender Representation in Multilingual Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Boundless Byte Pair Encoding: Breaking the Pre-tokenization Barriercited by 32
  2. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 11
  3. 2025Hebrew Diacritics Restoration using Visual Representationcited by 0
  4. 2025Diacritic Restoration for Low-Resource Indigenous Languages: Case Study with Bribri and Cook Islands M=aoricited by 0
  5. 2026The Degree of Language Diacriticity and Its Effect on Taskscited by 0
  6. 2026textitVersteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Languagecited by 0
  7. 2026The Functionalizer: Lossless Functional Decomposition for Subword Tokenizationcited by 0