mlsanityData to 14 Oct 2026

LLM-based phoneme-to-grapheme for phoneme-based speech recognition

arXiv 2506.04711 ↗First posted 5 Jun 2025cs.SD cs.CL eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

In automatic speech recognition (ASR), phoneme-based multilingual pre-training and crosslingual fine-tuning is attractive for its high data efficiency and competitive results compared to subword-based models. However, Weighted Finite State Transducer (WFST) based decoding is limited by its complex pipeline and inability to leverage large language models (LLMs). Therefore, we propose LLM-based phoneme-to-grapheme (LLM-P2G) decoding for phoneme-based ASR, consisting of speech-to-phoneme (S2P) and phoneme-to-grapheme (P2G). A challenge is that there seems to have information loss in cascading S2P and P2G. To address this challenge, we propose two training strategies: data augmentation with noisy phonemes (DANP), and randomized top-K marginalized (TKM) training and decoding. Our experimental results show that LLM-P2G outperforms WFST-based systems in crosslingual ASR for Polish and German, by relative WER reductions of 3.6% and 6.9% respectively.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

22 of 27 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022Emergent Abilities of Large Language Modelscited by 2,512
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  9. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  10. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  11. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 88
  12. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
10 more
  1. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  2. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 57
  3. 2024Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervisioncited by 17
  4. 2022Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilitiescited by 16
  5. 2020CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latencycited by 13
  6. 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
  7. 2021Multilingual and crosslingual speech recognition using phonological-vector based phone embeddingscited by 11
  8. 2023Optimizing Two-Pass Cross-Lingual Transfer Learning: Phoneme Recognition and Phoneme to Grapheme Translationcited by 5
  9. 2023TranUSR: Phoneme-to-word Transcoder Based Unified Speech Representation Learning for Cross-lingual Speech Recognitioncited by 4
  10. 2024Low-Resourced Speech Recognition for Iu Mien Language via Weakly-Supervised Phoneme-based Multilingual Pre-trainingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximationcited by 3
  2. 2026Advancing LLM-based phoneme-to-grapheme for multilingual speech recognitioncited by 0
  3. 2026Latent Softmax for Data-Efficient Phoneme-Based Multilingual ASR Across Tonal and Non-Tonal Languagescited by 0