mlsanityData to 20 Oct 2026

Investigating the Reasonable Effectiveness of Speaker Pre-Trained Models and their Synergistic Power for SingMOS Prediction

arXiv 2506.02232 ↗First posted 2 Jun 2025eess.AS cs.SDcited by 1

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

In this study, we focus on Singing Voice Mean Opinion Score (SingMOS) prediction. Previous research have shown the performance benefit with the use of state-of-the-art (SOTA) pre-trained models (PTMs). However, they haven't explored speaker recognition speech PTMs (SPTMs) such as x-vector, ECAPA and we hypothesize that it will be the most effective for SingMOS prediction. We believe that due to their speaker recognition pre-training, it equips them to capture fine-grained vocal features (e.g., pitch, tone, intensity) from synthesized singing voices in a much more better way than other PTMs. Our experiments with SOTA PTMs including SPTMs and music PTMs validates the hypothesis. Additionally, we introduce a novel fusion framework, BATCH that uses Bhattacharya Distance for fusion of PTMs. Through BATCH with the fusion of speaker recognition SPTMs, we report the topmost performance comparison to all the individual PTMs and baseline fusion techniques as well as setting SOTA.

Key facts. Cited by 1 paper in the map (data through 20 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 19 of its 24 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

19 of 24 references.

References · 19

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,671
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,388
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,137
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 984
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 679
  6. 2023Scaling Speech Technology to 1,000+ Languagescited by 484
  7. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 192
  8. 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 69
  9. 2022SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesiscited by 25
  10. 2022MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learningcited by 25
  11. 2024Heterogeneity over Homogeneity: Investigating Multilingual Speech Pre-Trained Models for Detecting Audio Deepfakecited by 21
  12. 2024MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Modelscited by 21
7 more
  1. 2023The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domainscited by 17
  2. 2023A Study on Incorporating Whisper for Robust Speech Assessmentcited by 16
  3. 2024SingMOS: An extensive Open-Source Singing Voice Dataset for MOS Predictioncited by 16
  4. 2022Fusion of Self-supervised Learned Models for MOS Predictioncited by 13
  5. 2022Investigation of Ensemble features of Self-Supervised Pretrained Models for Automatic Speech Recognitioncited by 13
  6. 2022Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Featurescited by 5
  7. 2023LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancementcited by 5

Built on by

Cited by year (2026 to 20 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Indic-CodecFake meets SATYAM: Towards Detecting Neural Audio Codec Synthesized Speech Deepfakes in Indic Languagescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction
  2. 2024Pitch-and-Spectrum-Aware Singing Quality Assessment with Bias Correction and Model Fusion
  3. 2021The Use of Voice Source Features for Sung Speech Recognition
  4. 2024Investigating Prosodic Signatures via Speech Pre-Trained Models for Audio Deepfake Source Attribution
  5. 2022The VoiceMOS Challenge 2022
  6. 2024Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models
  7. 2023A Systematic Exploration of Joint-training for Singing Voice Synthesis
  8. 2025Adapting Speech Language Model to Singing Voice Synthesis