# ALO-VC: Any-to-any Low-latency One-shot Voice Conversion

arXiv: https://arxiv.org/abs/2306.01100 · HTML: https://mlsanity.com/paper/2306.01100/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme](https://mlsanity.com/paper/2109.13821/) (2021)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion](https://mlsanity.com/paper/2106.10132/) (2021)
- [FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention](https://mlsanity.com/paper/2010.14150/) (2020)
- [Non-Parallel Voice Conversion with Cyclic Variational Autoencoder](https://mlsanity.com/paper/1907.10185/) (2019)
- [The Sequence-to-Sequence Baseline for the Voice Conversion Challenge 2020: Cascading ASR and TTS](https://mlsanity.com/paper/2010.02434/) (2020)
- [FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion](https://mlsanity.com/paper/2104.06900/) (2021)
- [Baseline System of Voice Conversion Challenge 2020 with Cyclic Variational Autoencoder and Parallel WaveGAN](https://mlsanity.com/paper/2010.04429/) (2020)
- [High-Fidelity and Low-Latency Universal Neural Vocoder based on Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform Modeling](https://mlsanity.com/paper/2105.09856/) (2021)
- [The NU Voice Conversion System for the Voice Conversion Challenge 2020: On the Effectiveness of Sequence-to-sequence Models and Autoregressive Neural Vocoders](https://mlsanity.com/paper/2010.04446/) (2020)
- [MOSRA: Joint Mean Opinion Score and Room Acoustics Speech Quality Assessment](https://mlsanity.com/paper/2204.01345/) (2022)
- [FastVC: Fast Voice Conversion with non-parallel data](https://mlsanity.com/paper/2010.04185/) (2020)
- [Low-Latency Real-Time Non-Parallel Voice Conversion based on Cyclic Variational Autoencoder and Multiband WaveRNN with Data-Driven Linear Prediction](https://mlsanity.com/paper/2105.09858/) (2021)
- [AC-VC: Non-parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion](https://mlsanity.com/paper/2111.06601/) (2021)

## Built on by (most cited first, 2 of 2)

- [Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion](https://mlsanity.com/paper/2507.14534/) (2025)
- [Exposing Vulnerabilities in Explanation for Time Series Classifiers via Dual-Target Attacks](https://mlsanity.com/paper/2602.02763/) (2026)

## Abstract (arXiv)

This paper presents ALO-VC, a non-parallel low-latency one-shot phonetic posteriorgrams (PPGs) based voice conversion method. ALO-VC enables any-to-any voice conversion using only one utterance from the target speaker, with only 47.5 ms future look-ahead. The proposed hybrid signal processing and machine learning pipeline combines a pre-trained speaker encoder, a pitch predictor to predict the converted speech's prosody, and positional encoding to convey the phoneme's location information. We introduce two system versions: ALO-VC-R, which uses a pre-trained d-vector speaker encoder, and ALO-VC-E, which improves performance using the ECAPA-TDNN speaker encoder. The experimental results demonstrate both ALO-VC-R and ALO-VC-E can achieve comparable performance to non-causal baseline systems on the VCTK dataset and two out-of-domain datasets. Furthermore, both proposed systems can be deployed on a single CPU core with 55 ms latency and 0.78 real-time factor. Our demo is available online.