mlsanityData to 14 Oct 2026

Low-latency Real-time Voice Conversion on CPU

arXiv 2311.00873 ↗First posted 1 Nov 2023cs.SD cs.LG eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

We adapt the architectures of previous audio manipulation and generation neural networks to the task of real-time any-to-one voice conversion. Our resulting model, LLVC (Low-latency Low-resource Voice Conversion), has a latency of under 20ms at a bitrate of 16kHz and runs nearly 2.8x faster than real-time on a consumer CPU. LLVC uses both a generative adversarial architecture as well as knowledge distillation in order to attain this performance. To our knowledge LLVC achieves both the lowest resource usage as well as the lowest latency of any open-source voice conversion model. We provide open-source samples, code, and pretrained model weights at https://github.com/KoeAI/LLVC.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 1 Nov 2023.

Builds on

18 of 27 references.

References · 18

  1. 2020Knowledge Distillation: A Surveycited by 1,603
  2. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  3. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 192
  4. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 110
  5. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  6. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  7. 2022HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancementcited by 49
  8. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
  9. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  10. 2020F0-consistent many-to-many non-parallel voice conversion via conditional autoencodercited by 35
  11. 2023RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Musiccited by 30
  12. 2020Knowledge Distillation in Deep Learning and its Applicationscited by 29
6 more
  1. 2019Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modellingcited by 16
  2. 2020How Far Are We from Robust Voice Conversion: A Surveycited by 14
  3. 2022Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transformcited by 9
  4. 2023Perceive and predict: self-supervised speech representation based loss functions for speech enhancementcited by 8
  5. 2023QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversioncited by 6
  6. 2021Knowledge Distillation Leveraging Alternative Soft Targets from Non-Parallel Qualified Speech Datacited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026TVTSyn: Content-Synchronous Time-Varying Timbre for Streaming Voice Conversion and Anonymizationcited by 5
  2. 2025HarmonicAttack: An Adaptive Cross-Domain Audio Watermark Removalcited by 3
  3. 2026Lightweight and perceptually-guided voice conversion for electro-laryngeal speechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Ultra-Low Latency Speech Enhancement - A Comprehensive Study
  2. 2023DualVC: Dual-mode Voice Conversion using Intra-model Knowledge Distillation and Hybrid Predictive Coding
  3. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis
  4. 2024Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference
  5. 2024Modulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
  6. 2025FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation
  7. 2025MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows
  8. 2024A Survey of Deep Learning Audio Generation Methods