mlsanityData to 7 Oct 2026

VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech

arXiv 2401.14321 ↗First posted 25 Jan 2024eess.AS cs.SDcited by 23

Not on the map. Why?

Abstract (arXiv)

Recent TTS models with decoder-only Transformer architecture, such as SPEAR-TTS and VALL-E, achieve impressive naturalness and demonstrate the ability for zero-shot adaptation given a speech prompt. However, such decoder-only TTS models lack monotonic alignment constraints, sometimes leading to hallucination issues such as mispronunciation, word skipping and repeating. To address this limitation, we propose VALL-T, a generative Transducer model that introduces shifting relative position embeddings for input phoneme sequence, explicitly indicating the monotonic generation process while maintaining the architecture of decoder-only Transformer. Consequently, VALL-T retains the capability of prompt-based zero-shot adaptation and demonstrates better robustness against hallucinations with a relative reduction of 28.3% in the word error rate.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jan 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 11; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  2. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  3. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  4. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  5. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  6. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 37
  7. 2024RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesiscited by 26
  8. 2024Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decodingcited by 17
  9. 2024Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decodingcited by 10
  10. 2025Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesiscited by 10
  11. 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
  12. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 7
11 more
  1. 2025AniME: Adaptive Multi-Agent Planning for Long Animation Generationcited by 7
  2. 2024Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speechcited by 4
  3. 2025Voice Cloning: Comprehensive Surveycited by 4
  4. 2025Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignmentcited by 4
  5. 2025Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policycited by 4
  6. 2026OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Modelcited by 3
  7. 2025TTS-Transducer: End-to-End Speech Synthesis with Neural Transducercited by 1
  8. 2025Speech Recognition Model Improves Text-to-Speech Synthesis using Fine-Grained Rewardcited by 1
  9. 2024SegINR: Segment-wise Implicit Neural Representation for Sequence Alignment in Neural Text-to-Speechcited by 0
  10. 2026Experience-Calibrated Contrastive Decoding for Mitigating Hallucinations in LM-Based Text-to-Speechcited by 0
  11. 2026Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Preventioncited by 0