mlsanityData to 7 Oct 2026

ByT5 model for massively multilingual grapheme-to-phoneme conversion

arXiv 2204.03067 ↗First posted 6 Apr 2022cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

In this study, we tackle massively multilingual grapheme-to-phoneme conversion through implementing G2P models based on ByT5. We have curated a G2P dataset from various sources that covers around 100 languages and trained large-scale multilingual G2P models based on ByT5. We found that ByT5 operating on byte-level inputs significantly outperformed the token-based mT5 model in terms of multilingual G2P. Pairwise comparison with monolingual models in these languages suggests that multilingual ByT5 models generally lower the phone error rate by jointly learning from a variety of languages. The pretrained model can further benefit low resource G2P through zero-shot prediction on unseen languages or provides pretrained weights for finetuning, which helps the model converge to a lower phone error rate than randomly initialized weights. To facilitate future research on multilingual G2P, we make available our code and pretrained multilingual G2P models at: https://github.com/lingjzhu/CharsiuG2P.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 13 of its 36 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

13 of 36 references.

References · 13

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  5. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  7. 2020Transformer based Grapheme-to-Phoneme Conversioncited by 32
  8. 2019Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversioncited by 26
  9. 2015Sequence-to-Sequence Neural Net Models for Grapheme-to-Phoneme Conversioncited by 19
  10. 2017Massively Multilingual Neural Grapheme-to-Phoneme Conversioncited by 15
  11. 2020g2pM: A Neural Grapheme-to-Phoneme Conversion Package for Mandarin Chinese Based on a New Open Benchmark Datasetcited by 15
  12. 2016Jointly Learning to Align and Convert Graphemes to Phonemes with Neural Attention Modelscited by 10
1 more
  1. 2020One Model to Pronounce Them All: Multilingual Grapheme-to-Phoneme Conversion With a Transformer Ensemblecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 8; 2025: 14; 2026: 18.

49 papers.

Sort:

By area:

  1. 2025ACE-Step: A Step Towards Music Generation Foundation Modelcited by 54
  2. 2022ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Modelscited by 25
  3. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 18
  4. 2023XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speechcited by 16
  5. 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
  6. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  7. 2024LLM-Powered Grapheme-to-Phoneme Conversion: Benchmark and Case Studycited by 9
  8. 2023Speak While You Think: Streaming Speech Synthesis During Text Generationcited by 8
  9. 2023SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulationcited by 7
  10. 2025LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLMcited by 7
  11. 2025IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modelingcited by 6
  12. 2025LingGym: How Far Are LLMs from Thinking Like Field Linguists?cited by 6
37 more
  1. 2024Exploring the Benefits of Tokenization of Discrete Acoustic Unitscited by 4
  2. 2024PolyIPA -- Multilingual Phoneme-to-Grapheme Conversion Modelcited by 4
  3. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  4. 2024Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speechcited by 3
  5. 2025Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximationcited by 3
  6. 2025CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonesecited by 3
  7. 2026HuPER: A Human-Inspired Framework for Phonetic Perceptioncited by 3
  8. 2023Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordingscited by 2
  9. 2023Multilingual context-based pronunciation learning for Text-to-Speechcited by 2
  10. 2024Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Modelscited by 2
  11. 2023Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P) Transductioncited by 1
  12. 2024Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learningcited by 1
  13. 2024Efficiently Identifying Low-Quality Language Subsets in Multilingual Datasets: A Case Study on a Large-Scale Multilingual Audio Datasetcited by 1
  14. 2024AyutthayaAlpha: A Thai-Latin Script Transliteration Transformercited by 1
  15. 2025Cross-Lingual IPA Contrastive Learning for Zero-Shot NERcited by 1
  16. 2025What You Read Isn't What You Hear: Linguistic Sensitivity in Deepfake Speech Detectioncited by 1
  17. 2025LatPhon: Lightweight Multilingual G2P for Romance Languages and Englishcited by 1
  18. 2026ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversioncited by 1
  19. 2023Automatic Heteronym Resolution Pipeline Using RAD-TTS Alignerscited by 0
  20. 2025Phonikud: Overcoming Phonetic Underspecification for Hebrew Text-To-Speechcited by 0
  21. 2025MixedG2P-T5: G2P-free Speech Synthesis for Mixed-script texts using Speech Self-Supervised Learning and Language Modelcited by 0
  22. 2025OLaPh: Optimal Language Phonemizercited by 0
  23. 2026Symphonym: Universal Phonetic Embeddings for Cross-Script Toponym Matchingcited by 0
  24. 2026Extracting accent features in spoken Brazilian Portuguese without sociolinguistic labelscited by 0
  25. 2026Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinesecited by 0
  26. 2026UR-BERT: Scaling Text Encoders for Massively Multilingual TTS Through Universal Romanization and Speech Token Predictioncited by 0
  27. 2026PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translationcited by 0
  28. 2026SPARCLE: SPeaker-aware Aligned Representations via Contrastive Language Embeddingscited by 0
  29. 2026GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speechcited by 0
  30. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0
  31. 2026Correlation between prosody and pragmatics: A case study of the discourse marker h=al=a `now' in Persiancited by 0
  32. 2026FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelinescited by 0
  33. 2026Towards Stress-Aware Sentence-Level Filipino G2P With Weakly-Supervised ByT5 Fine-Tuningcited by 0
  34. 2026Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Datacited by 0
  35. 2026Phonemizing User-Generated Text: A Benchmark, Taxonomy, and Compositional Approachcited by 0
  36. 2026Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoringcited by 0
  37. 2026SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabiccited by 0