mlsanityData to 14 Oct 2026

Diacritics Restoration using BERT with Analysis on Czech language

arXiv 2105.11408 ↗First posted 24 May 2021cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose a new architecture for diacritics restoration based on contextualized embeddings, namely BERT, and we evaluate it on 12 languages with diacritics. Furthermore, we conduct a detailed error analysis on Czech, a morphologically rich language with a high level of diacritization. Notably, we manually annotate all mispredictions, showing that roughly 44% of them are actually not errors, but either plausible variants (19%), or the system corrections of erroneous data (25%). Finally, we categorize the real errors in detail. We release the code at https://github.com/ufal/bert-diacritics-restoration.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 8 of its 21 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

8 of 21 references.

References · 8

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  4. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
  5. 2018Attentive Sequence-to-Sequence Learning for Diacritic Restoration of Yor`ub'a Language Textcited by 11
  6. 2020Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritizationcited by 11
  7. 2019Efficient Convolutional Neural Networks for Diacritic Restorationcited by 9
  8. 2019Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NERcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 1; 2025: 1; 2026: 1.

6 papers.

  1. 2021Improving Arabic Diacritization by Learning to Diacritize and Translatecited by 6
  2. 2022Correcting diacritics and typos with a ByT5 transformer modelcited by 6
  3. 2024YAD: Leveraging T5 for Improved Automatic Diacritization of Yor`ub'a Textcited by 4
  4. 2022Dilated Convolutional Neural Networks for Lightweight Diacritics Restorationcited by 0
  5. 2025Hebrew Diacritics Restoration using Visual Representationcited by 0
  6. 2026The Degree of Language Diacriticity and Its Effect on Taskscited by 0