mlsanityData to 19 Oct 2026

FineCombo-TTS: Collaborative and Precise Controllable Speech Synthesis Using Text Descriptions and Reference Speech

arXiv 2606.19209 ↗First posted 17 Jun 2026cs.SDcited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

Controllable text-to-speech (TTS) has become a key research focus. However, methods based on either reference speech or text descriptions lack flexibility and precise control, and recent joint approaches remain loosely coupled, with speech modeling timbre and text controlling global style. We propose FineCombo-TTS, a unified framework for speech synthesis grounded in reference speech and guided by text descriptions, enabling flexible and precise control over acoustic attributes. Instead of explicit attribute disentanglement, we learn a unified acoustic representation and introduce a Conditional Flow Matching (CFM)-based Speech Variance Predictor to model fine-grained reference-to-target transformations guided by text descriptions. To support relative attribute control, we construct FineEdit, a structured paired dataset that explicitly encodes source-to-target attribute variations. Experiments demonstrate that our approach achieves flexible, precise, and expressive controllable TTS.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 28 of its 29 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

28 of 29 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning12 Jun
  2. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control3 Jun
  3. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  4. 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotations2 Feb
  5. 2023Improving Language Model-Based Zero-Shot Text-to-Speech Synthesis with Multi-Scale Acoustic Prompts21 Sep
  6. 2023TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models28 Aug
  7. 2023PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptions31 May
  8. 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus30 May
  9. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision7 Feb
  10. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt31 Jan
  11. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
  12. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptions22 Nov
1 more
  1. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation6 Jun

Other references · 15

  1. 2022Classifier-Free Diffusion Guidancecited by 5,991
  2. 2022Flow Matching for Generative Modelingcited by 5,905
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,067
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,184
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 947
  6. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 619
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 615
  8. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 589
  9. 2023Simple and Controllable Music Generationcited by 540
  10. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 146
  11. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 58
  12. 2024VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modellingcited by 43
3 more
  1. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 39
  2. 2023Latent Space Editing in Transformer-Based Flow Matchingcited by 33
  3. 2025FleSpeech: Flexibly Controllable Speech Generation with Various Promptscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FC-TTS: Style and Timbre Control in Zero-Shot Text-to-Speech with Disentangled Speech Representations
  2. 2026CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis
  3. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey
  4. 2026ReStyle-TTS: Relative and Continuous Style Control for Zero-Shot Speech Synthesis
  5. 2026Qwen-Audio-3.0-TTS: Freely Controllable and Highly Robust Speech Synthesis with Multi-Stage Training Paradigm
  6. 2022Fine-grained Noise Control for Multispeaker Speech Synthesis
  7. 2026MAGIC-TTS: Fine-Grained Controllable Speech Synthesis with Explicit Local Duration and Pause Control
  8. 2025In This Environment, As That Speaker: A Text-Driven Framework for Multi-Attribute Speech Conversion