mlsanityData to 14 Oct 2026

grapheme-kit: Grapheme-Level Metrics and Text Processing for Multilingual NLP

arXiv 2607.22456 ↗First posted 24 Jul 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Existing lexical distance, similarity, and evaluation metrics operate on Unicode code points, which can misrepresent errors in writing systems where a single grapheme is represented by multiple Unicode code points. We introduce grapheme-kit, an open-source Python library that extends these metrics to operate on grapheme clusters instead. The library also provides improved grapheme processing for Tamil and Sinhala, including accurate grapheme cluster identification and grapheme composition/decomposition utilities. Through an OCR case study, we demonstrate that grapheme-level metrics provide a more faithful evaluation of complex scripts.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 34 references that are in the map. First posted on arXiv 24 Jul 2026.

Builds on

13 of 34 references.

References · 13

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  2. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  3. 2020Evaluation of Text Generation: A Surveycited by 277
  4. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  5. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  6. 2019Survey on Publicly Available Sinhala Natural Language Processing Tools and Researchcited by 46
  7. 2022DEMETR: Diagnosing Evaluation Metrics for Translationcited by 36
  8. 2024Advocating Character Error Rate for Multilingual ASR Evaluationcited by 24
  9. 2024Egalitarian Language Representation in Language Models: It All Begins with Tokenizerscited by 20
  10. 2023string2string: A Modern Python Library for String-to-String Algorithmscited by 9
  11. 2025Adaptability of ASR Models on Low-Resource Language: A Comparative Study of Whisper and Wav2Vec-BERT on Banglacited by 6
  12. 2022Neural Grapheme-to-Phoneme Conversion with Pre-trained Grapheme Modelscited by 4
1 more
  1. 2025GraDeT-HTR: A Resource-Efficient Bengali Handwritten Text Recognition System utilizing Grapheme-based Tokenizer and Decoder-only Transformercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCRcited by 0