mlsanityData to 7 Oct 2026

XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speech

arXiv 2305.19709 ↗First posted 31 May 2023cs.CL cs.SD eess.AScited by 16

Not on the map. Why?

Abstract (arXiv)

We present XPhoneBERT, the first multilingual model pre-trained to learn phoneme representations for the downstream text-to-speech (TTS) task. Our XPhoneBERT has the same model architecture as BERT-base, trained using the RoBERTa pre-training approach on 330M phoneme-level sentences from nearly 100 languages and locales. Experimental results show that employing XPhoneBERT as an input phoneme encoder significantly boosts the performance of a strong neural TTS model in terms of naturalness and prosody and also helps produce fairly high-quality speech with limited training data. We publicly release our pre-trained XPhoneBERT with the hope that it would facilitate future research and downstream TTS applications for multiple languages. Our XPhoneBERT model is available at https://github.com/VinAIResearch/XPhoneBERT

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

24 of 35 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  9. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  10. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  11. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  12. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
12 more
  1. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  2. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  3. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  4. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  5. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  6. 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
  7. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  8. 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
  9. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 22
  10. 2017A Fast and Accurate Vietnamese Word Segmentercited by 21
  11. 2022Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speechcited by 13
  12. 2022DRSpeech: Degradation-Robust Text-to-Speech Synthesis with Frame-Level and Utterance-Level Acoustic Representation Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 6.

16 papers.

Sort:

By area:

  1. 2025LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLMcited by 7
  2. 2024Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfercited by 6
  3. 2024Small Language Models Also Work With Small Vocabularies: Probing the Linguistic Abilities of Grapheme- and Phoneme-Based Baby Llamascited by 5
  4. 2024An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarioscited by 4
  5. 2025Cross-Lingual IPA Contrastive Learning for Zero-Shot NERcited by 1
  6. 2025PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decodingcited by 1
  7. 2025Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and Englishcited by 1
  8. 20263DGesPolicy: Phoneme-Aware Holistic Co-Speech Gesture Generation Based on Action Controlcited by 1
  9. 2024Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languagescited by 0
  10. 2025LAPS-Diff: A Diffusion-Based Framework for Singing Voice Synthesis With Language Aware Prosody-Style Guided Learningcited by 0
  11. 2025Multi-task Pretraining for Enhancing Interpretable L2 Pronunciation Assessmentcited by 0
  12. 2026UR-BERT: Scaling Text Encoders for Massively Multilingual TTS Through Universal Romanization and Speech Token Predictioncited by 0
4 more
  1. 2026PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translationcited by 0
  2. 2026Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editingcited by 0
  3. 2026One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthographycited by 0
  4. 2026Align Then Reason: A Multimodal Lip-Sync Judge for Dubbingcited by 0