mlsanityData to 7 Oct 2026

Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions

arXiv 2301.08810 ↗First posted 20 Jan 2023cs.CL cs.SD eess.AScited by 22

Not on the map. Why?

Abstract (arXiv)

Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns. However, these models are usually word-level or sup-phoneme-level and jointly trained with phonemes, making them inefficient for the downstream TTS task where only phonemes are needed. In this work, we propose a phoneme-level BERT (PL-BERT) with a pretext task of predicting the corresponding graphemes along with the regular masked phoneme predictions. Subjective evaluations show that our phoneme-level BERT encoder has significantly improved the mean opinion scores (MOS) of rated naturalness of synthesized speech compared with the state-of-the-art (SOTA) StyleTTS baseline on out-of-distribution (OOD) texts.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 20 Jan 2023.

Builds on

19 of 25 references.

References · 19

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  9. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  10. 2021A Survey on Neural Speech Synthesiscited by 182
  11. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  12. 2018A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguisticscited by 64
7 more
  1. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  2. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  3. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  4. 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
  5. 2020Unified Mandarin TTS Front-end Based on Distilled BERT Modelcited by 18
  6. 2022Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speechcited by 13
  7. 2022StyleTTS-VC: One-Shot Voice Conversion by Knowledge Transfer from Style-Based TTS Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 7; 2026: 5.

22 papers.

Sort:

By area:

  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  2. 2023XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speechcited by 16
  3. 2024From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemescited by 11
  4. 2025Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbingcited by 9
  5. 2024Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfercited by 6
  6. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  7. 2025IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modelingcited by 6
  8. 2025SpeechWeave: Diverse Multilingual Synthetic Text & Audio Data Generation Pipeline for Training Text to Speech Modelscited by 5
  9. 2025Voice Cloning: Comprehensive Surveycited by 4
  10. 2025BabyLM's First Words: Word Segmentation as a Phonological Probing Taskcited by 2
  11. 2025Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioningcited by 2
  12. 2023Multi-Modal Automatic Prosody Annotation with Contrastive Pretraining of SSWPcited by 1
10 more
  1. 2024Llama-VITS: Enhancing TTS Synthesis with Semantic Awarenesscited by 1
  2. 2024PRESENT: Zero-Shot Text-to-Prosody Controlcited by 0
  3. 2024Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERTcited by 0
  4. 2024Word-wise intonation model for cross-language TTS systemscited by 0
  5. 2025Empowering Global Voices: A Data-Efficient, Phoneme-Tone Adaptive Approach to High-Fidelity Speech Synthesiscited by 0
  6. 2026MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Controlcited by 0
  7. 2026UR-BERT: Scaling Text Encoders for Massively Multilingual TTS Through Universal Romanization and Speech Token Predictioncited by 0
  8. 2026Adaptive Oscillatory Inductive Bias for Modeling Sharp Prosodic Dynamics in Diffusion-Based TTScited by 0
  9. 2026Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speechcited by 0
  10. 2026Continuous-Time Acoustic Modelling with Neural Controlled Differential Equationscited by 0