mlsanityData to 7 Oct 2026

Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion

arXiv 2507.14534 ↗First posted 19 Jul 2025eess.AS cs.CL cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Zero-shot online voice conversion (VC) holds significant promise for real-time communications and entertainment. However, current VC models struggle to preserve semantic fidelity under real-time constraints, deliver natural-sounding conversions, and adapt effectively to unseen speaker characteristics. To address these challenges, we introduce Conan, a chunkwise online zero-shot voice conversion model that preserves the content of the source while matching the voice timbre and styles of reference speech. Conan comprises three core components: 1) a Stream Content Extractor that leverages Emformer for low-latency streaming content encoding; 2) an Adaptive Style Encoder that extracts fine-grained stylistic features from reference speech for enhanced style adaptation; 3) a Causal Shuffle Vocoder that implements a fully causal HiFiGAN using a pixel-shuffle mechanism. Experimental evaluations demonstrate that Conan outperforms baseline models in subjective and objective metrics. Audio samples can be found at https://aaronz345.github.io/ConanDemo.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 19 Jul 2025.

Builds on

46 of 57 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2016Least Squares Generative Adversarial Networkscited by 1,000
  8. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  9. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  10. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  11. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  12. 2023Any-to-Any Generation via Composable Diffusioncited by 203
34 more
  1. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  2. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  3. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  4. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  5. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  6. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  7. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  8. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  9. 2023Online Clustered Codebookcited by 55
  10. 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codeccited by 47
  11. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  12. 2023Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesiscited by 41
  13. 2020HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesiscited by 39
  14. 2024GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Taskscited by 29
  15. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  16. 2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
  17. 2023StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesiscited by 21
  18. 2024TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Controlcited by 20
  19. 2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
  20. 2023LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Modelscited by 18
  21. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15
  22. 2024StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversioncited by 9
  23. 2024End-to-end streaming model for low-latency speech anonymizationcited by 9
  24. 2021Non-autoregressive sequence-to-sequence voice conversioncited by 7
  25. 2022ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speedcited by 7
  26. 2023QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversioncited by 6
  27. 2025ISDrama: Immersive Spatial Drama Generation through Multimodal Promptingcited by 6
  28. 2021FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversioncited by 5
  29. 2022Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidancecited by 5
  30. 2025TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesiscited by 5
  31. 2025Versatile Framework for Song Generation with Prompt-based Controlcited by 4
  32. 2023DualVC 2: Dynamic Masked Convolution for Unified Streaming and Non-Streaming Voice Conversioncited by 3
  33. 2023ALO-VC: Any-to-any Low-latency One-shot Voice Conversioncited by 2
  34. 2024CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System Based on Conditional Variational Autoencodercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026TVTSyn: Content-Synchronous Time-Varying Timbre for Streaming Voice Conversion and Anonymizationcited by 4
  2. 2026X-VC: Zero-shot Streaming Voice Conversion in Codec Spacecited by 2
  3. 2026Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformercited by 0
  4. 2026VOSSA: Voiceprint Optimization for Streaming Speech Architecturescited by 0