mlsanityData to 7 Oct 2026

Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning

arXiv 2506.04527 ↗First posted 5 Jun 2025cs.SD cs.CL eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

We propose a model to obtain phonemic and prosodic labels of speech that are coherent with graphemes. Unlike previous methods that simply fine-tune a pre-trained ASR model with the labels, the proposed model conditions the label generation on corresponding graphemes by two methods: 1) Add implicit grapheme conditioning through prompt encoder using pre-trained BERT features. 2) Explicitly prune the label hypotheses inconsistent with the grapheme during inference. These methods enable obtaining parallel data of speech, the labels, and graphemes, which is applicable to various downstream tasks such as text-to-speech and accent estimation from text. Experiments showed that the proposed method significantly improved the consistency between graphemes and the predicted labels. Further, experiments on accent estimation task confirmed that the created parallel data by the proposed method effectively improve the estimation accuracy.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

28 of 42 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  7. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  8. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  9. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  10. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  11. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 56
  12. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
16 more
  1. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  2. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
  3. 2017Listening while Speaking: Speech Chain by Deep Learningcited by 46
  4. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  5. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 34
  6. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 22
  7. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  8. 2024Release of Pre-Trained Models for the Japanese Languagecited by 22
  9. 2018Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent languagecited by 14
  10. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  11. 2017Building competitive direct acoustics-to-word models for English conversational speech recognitioncited by 10
  12. 2019A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesiscited by 8
  13. 2022Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesiscited by 6
  14. 2022Polyphone disambiguation and accent prediction using pre-trained language models in Japanese TTS front-endcited by 5
  15. 2024Audio-conditioned phonemic and prosodic annotation for building text-to-speech models from unlabeled speech datacited by 3
  16. 2024Improving Speech Recognition Error Prediction for Modern and Off-the-shelf Speech Recognizerscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Building Tailored Speech Recognizers for Japanese Speaking Assessmentcited by 1
  2. 2026Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoringcited by 0