# Mel-spectrogram augmentation for sequence to sequence voice conversion

arXiv: https://arxiv.org/abs/2001.01401 · HTML: https://mlsanity.com/paper/2001.01401/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2020.

## Built on by (most cited first, 7 of 7)

- [Msanii: High Fidelity Music Synthesis on a Shoestring Budget](https://mlsanity.com/paper/2301.06468/) (2023)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [A Mel Spectrogram Enhancement Paradigm Based on CWT in Speech Synthesis](https://mlsanity.com/paper/2406.12164/) (2024)
- [Discrete Unit based Masking for Improving Disentanglement in Voice Conversion](https://mlsanity.com/paper/2409.11560/) (2024)
- [An Investigation of Test-time Adaptation for Audio Classification under Background Noise](https://mlsanity.com/paper/2507.15523/) (2025)
- [AMAuT: A Flexible and Efficient Multiview Audio Transformer Framework Trained from Scratch](https://mlsanity.com/paper/2510.19368/) (2025)
- [DHAuDS: A Dynamic and Heterogeneous Audio Benchmark for Test-Time Adaptation](https://mlsanity.com/paper/2511.18421/) (2025)

## Abstract (arXiv)

For training the sequence-to-sequence voice conversion model, we need to handle an issue of insufficient data about the number of speech pairs which consist of the same utterance. This study experimentally investigated the effects of Mel-spectrogram augmentation on training the sequence-to-sequence voice conversion (VC) model from scratch. For Mel-spectrogram augmentation, we adopted the policies proposed in SpecAugment. In addition, we proposed new policies (i.e., frequency warping, loudness and time length control) for more data variations. Moreover, to find the appropriate hyperparameters of augmentation policies without training the VC model, we proposed hyperparameter search strategy and the new metric for reducing experimental cost, namely deformation per deteriorating ratio. We compared the effect of these Mel-spectrogram augmentation methods based on various sizes of training set and augmentation policies. In the experimental results, the time axis warping based policies (i.e., time length control and time warping.) showed better performance than other policies. These results indicate that the use of the Mel-spectrogram augmentation is more beneficial for training the VC model.