mlsanityData to 14 Oct 2026

UR-BERT: Scaling Text Encoders for Massively Multilingual TTS Through Universal Romanization and Speech Token Prediction

arXiv 2606.11681 ↗First posted 10 Jun 2026cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose UR-BERT, a Romanized transcription-based text-to-speech (TTS) encoder for massively multilingual TTS systems. Conventional grapheme-to-phoneme (G2P)-based approaches are limited to around 100 languages due to the availability of reliable G2P resources. In contrast, UR-BERT scales to 495 languages by unifying diverse writing systems into a shared Romanization representation. To further enhance phonetic fidelity and text-speech alignment, we introduce a speech token prediction objective during training, which encourages the encoder to learn speech-aware phonetic representations in a data-efficient manner. Experiments show that TTS systems built on UR-BERT consistently outperform recent text encoder baselines across a wide range of languages and resource conditions, and demonstrate strong generalization to unseen languages.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

40 of 53 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  8. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  9. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  10. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  11. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  12. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
28 more
  1. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  2. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  3. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  4. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  6. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  7. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  8. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  9. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  10. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 169
  11. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  12. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 157
  13. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  14. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
  15. 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
  16. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  17. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  18. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 50
  19. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  20. 2023SELM: Speech Enhancement Using Discrete Tokens and Language Modelscited by 36
  21. 2025Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languagescited by 34
  22. 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
  23. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 23
  24. 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Predictioncited by 17
  25. 2023XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speechcited by 16
  26. 2022Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speechcited by 13
  27. 2024LAMA-UT: Language Agnostic Multilingual ASR through Orthography Unification and Language-Specific Transliterationcited by 3
  28. 2025Differentiable K-means for Fully-optimized Discrete Token-based ASRcited by 3

Built on by

Not cited yet.