mlsanityData to 7 Oct 2026

Transduce and Speak: Neural Transducer for Text-to-Speech with Semantic Token Prediction

arXiv 2311.02898 ↗First posted 6 Nov 2023eess.AS cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce a text-to-speech(TTS) framework based on a neural transducer. We use discretized semantic tokens acquired from wav2vec2.0 embeddings, which makes it easy to adopt a neural transducer for the TTS framework enjoying its monotonic alignment constraints. The proposed model first generates aligned semantic tokens using the neural transducer, then synthesizes a speech sample from the semantic tokens using a non-autoregressive(NAR) speech generator. This decoupled framework alleviates the training complexity of TTS and allows each stage to focus on 1) linguistic and alignment modeling and 2) fine-grained acoustic modeling, respectively. Experimental results on the zero-shot adaptive TTS show that the proposed model exceeds the baselines in speech quality and speaker similarity via objective and subjective measures. We also investigate the inference speed and prosody controllability of our proposed model, showing the potential of the neural transducer for TTS frameworks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

18 of 23 references.

References · 18

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  8. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  9. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  10. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  11. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  12. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
6 more
  1. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  2. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
  3. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  4. 2022Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpuscited by 10
  5. 2020Visually Guided Self Supervised Learning of Speech Representationscited by 8
  6. 2019Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignmentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
  2. 2025Zero-Shot Streaming Text to Speech Synthesis with Transducer and Auto-Regressive Modelingcited by 3
  3. 2024Utilizing Neural Transducers for Two-Stage Text-to-Speech via Semantic Token Predictioncited by 1
  4. 2024Transducer Consistency Regularization for Speech to Text Applicationscited by 1
  5. 2025TTS-Transducer: End-to-End Speech Synthesis with Neural Transducercited by 1
  6. 2026Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Inputcited by 1
  7. 2024SegINR: Segment-wise Implicit Neural Representation for Sequence Alignment in Neural Text-to-Speechcited by 0