mlsanityData to 14 Oct 2026

MixedG2P-T5: G2P-free Speech Synthesis for Mixed-script texts using Speech Self-Supervised Learning and Language Model

arXiv 2509.01391 ↗First posted 1 Sep 2025eess.AS cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This study presents a novel approach to voice synthesis that can substitute the traditional grapheme-to-phoneme (G2P) conversion by using a deep learning-based model that generates discrete tokens directly from speech. Utilizing a pre-trained voice SSL model, we train a T5 encoder to produce pseudo-language labels from mixed-script texts (e.g., containing Kanji and Kana). This method eliminates the need for manual phonetic transcription, reducing costs and enhancing scalability, especially for large non-transcribed audio datasets. Our model matches the performance of conventional G2P-based text-to-speech systems and is capable of synthesizing speech that retains natural linguistic and paralinguistic features, such as accents and intonations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

21 of 28 references.

References · 21

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  10. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  11. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  12. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
9 more
  1. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  2. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 58
  3. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  4. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 22
  5. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  6. 2020One Model to Pronounce Them All: Multilingual Grapheme-to-Phoneme Conversion With a Transformer Ensemblecited by 4
  7. 2020Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversioncited by 4
  8. 2022J-MAC: Japanese multi-speaker audiobook corpus for speech synthesiscited by 4
  9. 2020JSSS: free Japanese speech corpus for summarization and simplificationcited by 3

Built on by

Not cited yet.