mlsanityData to 14 Oct 2026

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

arXiv 2609.30924 ↗First posted 25 Sep 2026cs.CL cs.SD eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate and efficient pronunciation transcription is essential for preparing text-to-speech training data at scale. Existing approaches have different limitations: grapheme-to-pronunciation (G2P) and speech-to-pronunciation (S2P) methods each capture only partial information, using only text or only speech, while speech-and-text-to-pronunciation (ST2P) methods use both but require costly pronunciation-annotated data. To address this problem, we propose a training-free ST2P pipeline that integrates both lexical and acoustic information at inference time. Lexical resources and G2P tools generate text-constrained candidates, and a left-to-right greedy search selects the best one using whole-sequence negative log-likelihoods from frozen pretrained S2P models. On three Japanese corpora, our method reduces Character Error Rate (CER) from 0.60--1.40% (text-only baseline) to 0.04--0.17% with reference transcripts, and 0.64--1.58% with ASR transcripts. It outperforms all baselines, including a trained ST2P model and commercial multimodal LLMs. Our greedy search method is 3--3.5× faster than beam search at similar CER, and the cascade is 2× faster than direct decoding ensuring the efficiency and accuracy. In Spanish, French, and preliminary English, it also surpasses four open multimodal LLMs and the best traditional methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

16 of 28 references.

References · 16

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2025Qwen2.5-Omni Technical Reportcited by 736
  4. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  6. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  7. 2025Qwen3-Omni Technical Reportcited by 497
  8. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 58
  9. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  10. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 22
  11. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  12. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
4 more
  1. 2026Benchmarking Large Language Models for Grapheme-to-Phoneme Conversion: A Japanese Case Studycited by 3
  2. 2026Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesiscited by 3
  3. 2025Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioningcited by 2
  4. 2025Building Tailored Speech Recognizers for Japanese Speaking Assessmentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Data-efficient Targeted Token-level Preference Optimization for LLM-based Text-to-Speech
  2. 2025CLiFT-ASR: A Cross-Lingual Fine-Tuning Framework for Low-Resource Taiwanese Hokkien Speech Recognition
  3. 2026ZONOS2 Technical Report
  4. 2022Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translation
  5. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
  6. 2026Simultaneous Speech-to-Speech Translation Without Aligned Data
  7. 2025Pronunciation Editing for Finnish Speech using Phonetic Posteriorgrams
  8. 2025JCAPT: A Joint Modeling Approach for CAPT