mlsanityData to 7 Oct 2026

StyleStream: Real-Time Zero-Shot Voice Style Conversion

arXiv 2602.20113 ↗First posted 23 Feb 2026cs.SD cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 58 of its 68 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

58 of 68 references.

References · 58

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2022A ConvNet for the 2020scited by 3,545
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  10. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  11. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  12. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
38 more
  1. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  2. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  3. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  4. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  5. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  6. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  7. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  8. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  9. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
  10. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  11. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 150
  12. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
  13. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  14. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  15. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 108
  16. 2020Unsupervised Speech Decomposition via Triple Information Bottleneckcited by 87
  17. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  18. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  19. 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
  20. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  21. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
  22. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  23. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  24. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  25. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 39
  26. 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
  27. 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
  28. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  29. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  30. 2025Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglementcited by 34
  31. 2024GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speechcited by 30
  32. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  33. 2022SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottleneckscited by 27
  34. 2023CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voicecited by 19
  35. 2020VAW-GAN for Disentanglement and Recomposition of Emotional Elements in Speechcited by 18
  36. 2024Coding Speech through Vocal Tract Kinematicscited by 18
  37. 2022Voice-preserving Zero-shot Multiple Accent Conversioncited by 17
  38. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training
  2. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
  3. 2025Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
  4. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion
  5. 2025MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows
  6. 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Prompts
  7. 2025Text-Driven Voice Conversion via Latent State-Space Modeling ↗
  8. 2024Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling