mlsanityData to 14 Oct 2026

AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis

arXiv 2504.10309 ↗First posted 14 Apr 2025cs.SD cs.AIcited by 1

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

With the advancement of speech synthesis technology, users have higher expectations for the naturalness and expressiveness of synthesized speech. But previous research ignores the importance of prompt selection. This study proposes a text-to-speech (TTS) framework based on Retrieval-Augmented Generation (RAG) technology, which can dynamically adjust the speech style according to the text content to achieve more natural and vivid communication effects. We have constructed a speech style knowledge database containing high-quality speech samples in various contexts and developed a style matching scheme. This scheme uses embeddings, extracted by Llama, PER-LLM-Embedder,and Moka, to match with samples in the knowledge database, selecting the most appropriate speech style for synthesis. Furthermore, our empirical research validates the effectiveness of the proposed method. Our demo can be viewed at: https://thuhcsi.github.io/icme2025-AutoStyle-TTS

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 28 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

22 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Retrieval-Augmented Text-to-Audio Generation14 Sep

Other references · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  5. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  6. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  7. 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 102
  8. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  9. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
  10. 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
  11. 2024VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modellingcited by 21
  12. 2024SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Modelscited by 17
9 more
  1. 2018Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexacited by 11
  2. 2024Retrieval Augmented Generation in Prompt-based Text-to-Speech Synthesis with Context-Aware Contrastive Language-Audio Pretrainingcited by 9
  3. 2024Controlling Emotion in Text-to-Speech with Natural Language Promptscited by 9
  4. 2024RAG-based Question Answering over Heterogeneous Data and Textcited by 7
  5. 2022Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contextscited by 6
  6. 2015Clustering is Efficient for Approximate Maximum Inner Product Searchcited by 3
  7. 2023U-Style: Cascading U-nets with Multi-level Speaker and Style Modeling for Zero-Shot Voice Cloningcited by 2
  8. 2024StyleFusion TTS: Multimodal Style-control and Enhanced Feature Fusion for Zero-shot Text-to-speech Synthesiscited by 2
  9. 2024Expressive Prompting: Improving Emotion Intensity and Speaker Consistency in Zero-Shot TTScited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Modeling Sarcastic Speech: Semantic and Prosodic Cues in a Speech Synthesis Frameworkcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesis
  2. 2024Style Mixture of Experts for Expressive Text-To-Speech Synthesis
  3. 2024AS-Speech: Adaptive Style For Speech Synthesis ↗
  4. 2025HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesis
  5. 2026Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models
  6. 2023Stylebook: Content-Dependent Speaking Style Modeling for Any-to-Any Voice Conversion using Only Speech Data
  7. 2022NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTS
  8. 2025Retrieval-Augmented Dialogue Knowledge Aggregation for Expressive Conversational Speech Synthesis