mlsanityData to 14 Oct 2026

Voice Impression Control in Zero-Shot TTS

arXiv 2506.05688 ↗First posted 6 Jun 2025cs.SD cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Para-/non-linguistic information in speech is pivotal in shaping the listeners' impression. Although zero-shot text-to-speech (TTS) has achieved high speaker fidelity, modulating subtle para-/non-linguistic information to control perceived voice characteristics, i.e., impressions, remains challenging. We have therefore developed a voice impression control method in zero-shot TTS that utilizes a low-dimensional vector to represent the intensities of various voice impression pairs (e.g., dark-bright). The results of both objective and subjective evaluations have demonstrated our method's effectiveness in impression control. Furthermore, generating this vector via a large language model enables target-impression generation from a natural language description of the desired impression, thus eliminating the need for manual optimization. Audio examples are available on our demo page (https://ntt-hilab-gensp.github.io/is2025voiceimpression/).

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

28 of 37 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  5. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  6. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  9. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  10. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 243
  11. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  12. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
16 more
  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 105
  2. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
  3. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  4. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  5. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27
  6. 2015Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problemscited by 23
  7. 2021Cross-speaker emotion disentangling and transfer for end-to-end speech synthesiscited by 21
  8. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
  9. 2024LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioningcited by 15
  10. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  11. 2023Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation modelcited by 10
  12. 2021Cross-speaker Style Transfer with Prosody Bottleneck in Neural Speech Synthesiscited by 9
  13. 2021Model architectures to extrapolate emotional expressions in DNN-based text-to-speechcited by 6
  14. 2019Robust speaker recognition using unsupervised adversarial invariancecited by 4
  15. 2025Speech Synthesis along Perceptual Voice Quality Dimensionscited by 4
  16. 2023Controllable Speaking Styles Using a Large Language Modelcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Investigation for Relative Voice Impression Estimationcited by 1
  2. 2026Scalable Direction-Following TTS via Voice Impression-Guided Pseudo Triplet Constructioncited by 0
  3. 2026ReaFlow-TTS: Realization-Conditioned Flow Matching for High-Quality and Controllable Speech Synthesiscited by 0