# An Adaptive Learning based Generative Adversarial Network for One-To-One Voice Conversion arXiv: https://arxiv.org/abs/2104.12159 ยท HTML: https://mlsanity.com/paper/2104.12159/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 33 of its 68 references that are in the map. - First posted on arXiv 25 Apr 2021. ## Builds on (most cited first, 30 of 33) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016) - [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016) - [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016) - [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016) - [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016) - [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019) - [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017) - [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020) - [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020) - [StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks](https://mlsanity.com/paper/1806.02169/) (2018) - [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020) - [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018) - [CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion](https://mlsanity.com/paper/1904.04631/) (2019) - [CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversion](https://mlsanity.com/paper/2010.11672/) (2020) - [Cotatron: Transcription-Guided Speech Encoder for Any-to-Many Voice Conversion without Parallel Data](https://mlsanity.com/paper/2005.03295/) (2020) - [MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrograms](https://mlsanity.com/paper/1910.03713/) (2019) - [WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement](https://mlsanity.com/paper/2004.04098/) (2020) - [Non-parallel Emotion Conversion using a Deep-Generative Hybrid Network and an Adversarial Pair Discriminator](https://mlsanity.com/paper/2007.12932/) (2020) - [Mic2Mic: Using Cycle-Consistent Generative Adversarial Networks to Overcome Microphone Variability in Speech Systems](https://mlsanity.com/paper/2003.12425/) (2020) - [A Multi-Discriminator CycleGAN for Unsupervised Non-Parallel Speech Domain Adaptation](https://mlsanity.com/paper/1804.00522/) (2018) - [Many-to-Many Voice Conversion using Conditional Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/2002.06328/) (2020) - [ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks](https://mlsanity.com/paper/1905.11173/) (2019) - [Nonparallel Voice Conversion with Augmented Classifier Star Generative Adversarial Networks](https://mlsanity.com/paper/2008.12604/) (2020) - [Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN](https://mlsanity.com/paper/2008.04562/) (2020) - [A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment](https://mlsanity.com/paper/1804.08438/) (2018) - [Rhythm-Flexible Voice Conversion without Parallel Data Using Cycle-GAN over Phoneme Posteriorgram Sequences](https://mlsanity.com/paper/1808.03113/) (2018) - [WaveCycleGAN2: Time-domain Neural Post-filter for Speech Waveform Generation](https://mlsanity.com/paper/1904.02892/) (2019) - [CVC: Contrastive Learning for Non-parallel Voice Conversion](https://mlsanity.com/paper/2011.00782/) (2020) ## Built on by (most cited first, 3 of 3) - [Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion](https://mlsanity.com/paper/2311.08104/) (2023) - [Generative Adversarial Network based Voice Conversion: Techniques, Challenges, and Recent Advancements](https://mlsanity.com/paper/2504.19197/) (2025) - [Collective Learning Mechanism based Optimal Transport Generative Adversarial Network for Non-parallel Voice Conversion](https://mlsanity.com/paper/2504.13791/) (2025) ## Abstract (arXiv) Voice Conversion (VC) emerged as a significant domain of research in the field of speech synthesis in recent years due to its emerging application in voice-assisting technology, automated movie dubbing, and speech-to-singing conversion to name a few. VC basically deals with the conversion of vocal style of one speaker to another speaker while keeping the linguistic contents unchanged. VC task is performed through a three-stage pipeline consisting of speech analysis, speech feature mapping, and speech reconstruction. Nowadays the Generative Adversarial Network (GAN) models are widely in use for speech feature mapping from source to target speaker. In this paper, we propose an adaptive learning-based GAN model called ALGAN-VC for an efficient one-to-one VC of speakers. Our ALGAN-VC framework consists of some approaches to improve the speech quality and voice similarity between source and target speakers. The model incorporates a Dense Residual Network (DRN) like architecture to the generator network for efficient speech feature learning, for source to target speech feature conversion. We also integrate an adaptive learning mechanism to compute the loss function for the proposed model. Moreover, we use a boosted learning rate approach to enhance the learning capability of the proposed model. The model is trained by using both forward and inverse mapping simultaneously for a one-to-one VC. The proposed model is tested on Voice Conversion Challenge (VCC) 2016, 2018, and 2020 datasets as well as on our self-prepared speech dataset, which has been recorded in Indian regional languages and in English. A subjective and objective evaluation of the generated speech samples indicated that the proposed model elegantly performed the voice conversion task by achieving high speaker similarity and adequate speech quality.