# Voice Conversion Augmentation for Speaker Recognition on Defective Datasets

arXiv: https://arxiv.org/abs/2404.00863 · HTML: https://mlsanity.com/paper/2404.00863/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2024.

## Built on by (most cited first, 4 of 4)

- [EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis](https://mlsanity.com/paper/2608.23791/) (2026)
- [Improving Low-Resource Dialect Classification Using Retrieval-based Voice Conversion](https://mlsanity.com/paper/2507.03641/) (2025)
- [Interpolating Speaker Identities in Embedding Space for Data Expansion](https://mlsanity.com/paper/2508.19210/) (2025)
- [Bridging the Language Gap: Synthetic Voice Diversity via Latent Mixup for Equitable Speech Recognition](https://mlsanity.com/paper/2511.20534/) (2025)

## Abstract (arXiv)

Modern speaker recognition system relies on abundant and balanced datasets for classification training. However, diverse defective datasets, such as partially-labelled, small-scale, and imbalanced datasets, are common in real-world applications. Previous works usually studied specific solutions for each scenario from the algorithm perspective. However, the root cause of these problems lies in dataset imperfections. To address these challenges with a unified solution, we propose the Voice Conversion Augmentation (VCA) strategy to obtain pseudo speech from the training set. Furthermore, to guarantee generation quality, we designed the VCA-NN~(nearest neighbours) strategy to select source speech from utterances that are close to the target speech in the representation space. Our experimental results on three created datasets demonstrated that VCA-NN effectively mitigates these dataset problems, which provides a new direction for handling the speaker recognition problems from the data aspect.