mlsanityData to 7 Oct 2026

SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Models

arXiv 2406.08445 ↗First posted 12 Jun 2024eess.AS cs.LG cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Representations from pre-trained speech foundation models (SFMs) have shown impressive performance in many downstream tasks. However, the potential benefits of incorporating pre-trained SFM representations into speaker voice similarity assessment have not been thoroughly investigated. In this paper, we propose SVSNet+, a model that integrates pre-trained SFM representations to improve performance in assessing speaker voice similarity. Experimental results on the Voice Conversion Challenge 2018 and 2020 datasets show that SVSNet+ incorporating WavLM representations shows significant improvements compared to baseline models. In addition, while fine-tuning WavLM with a small dataset of the downstream task does not improve performance, using the same dataset to learn a weighted-sum representation of WavLM can substantially improve performance. Furthermore, when WavLM is replaced by other SFMs, SVSNet+ still outperforms the baseline models and exhibits strong generalization ability.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 27 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

24 of 27 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  7. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  8. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  9. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  10. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  11. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 71
  12. 2021Generalization Ability of MOS Prediction Networkscited by 69
12 more
  1. 2018The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methodscited by 68
  2. 2022Audio Self-supervised Learning: A Surveycited by 65
  3. 2023TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorchcited by 25
  4. 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
  5. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20
  6. 2020Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissionscited by 19
  7. 2023A Study on Incorporating Whisper for Robust Speech Assessmentcited by 11
  8. 2022Fusion of Self-supervised Learned Models for MOS Predictioncited by 10
  9. 2022Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applicationscited by 7
  10. 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
  11. 2021SVSNet: An End-to-end Speaker Voice Similarity Assessment Modelcited by 4
  12. 2023Mandarin Electrolaryngeal Speech Voice Conversion using Cross-domain Featurescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Acoustic and perceptual differences between standard and accented speech and their voice clonescited by 0