# Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024

arXiv: https://arxiv.org/abs/2409.02302 · HTML: https://mlsanity.com/paper/2409.02302/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 42 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 30 of 35)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofing](https://mlsanity.com/paper/2111.04433/) (2021)
- [One-class Learning Towards Synthetic Voice Spoofing Detection](https://mlsanity.com/paper/2010.13995/) (2020)
- [The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance](https://mlsanity.com/paper/2204.05177/) (2022)
- [The Vicomtech Audio Deepfake Detection System based on Wav2Vec2 for the 2022 ADD Challenge](https://mlsanity.com/paper/2203.01573/) (2022)
- [Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection](https://mlsanity.com/paper/2406.17376/) (2024)
- [ASSERT: Anti-Spoofing with Squeeze-Excitation and Residual neTworks](https://mlsanity.com/paper/1904.01120/) (2019)
- [Adversarial Attacks on Spoofing Countermeasures of automatic speaker verification](https://mlsanity.com/paper/1910.08716/) (2019)
- [SingFake: Singing Voice Deepfake Detection](https://mlsanity.com/paper/2309.07525/) (2023)
- [Can large-scale vocoded spoofed data improve speech spoofing countermeasure with a self-supervised front end?](https://mlsanity.com/paper/2309.06014/) (2023)
- [Attentive Merging of Hidden Embeddings from Pre-trained Speech Model for Anti-spoofing Detection](https://mlsanity.com/paper/2406.10283/) (2024)
- [CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection](https://mlsanity.com/paper/2406.02438/) (2024)
- [CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems](https://mlsanity.com/paper/2406.07237/) (2024)
- [JVS-MuSiC: Japanese multispeaker singing-voice corpus](https://mlsanity.com/paper/2001.07044/) (2020)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [MFA: TDNN with Multi-scale Frequency-channel Attention for Text-independent Speaker Verification with Short Utterances](https://mlsanity.com/paper/2202.01624/) (2022)
- [FSD: An Initial Chinese Dataset for Fake Song Detection](https://mlsanity.com/paper/2309.02232/) (2023)
- [Improving Short Utterance Anti-Spoofing with AASIST2](https://mlsanity.com/paper/2309.08279/) (2023)
- [How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?](https://mlsanity.com/paper/2406.02483/) (2024)
- [Singing Voice Graph Modeling for SingFake Detection](https://mlsanity.com/paper/2406.03111/) (2024)
- [Ensemble Models for Spoofing Detection in Automatic Speaker Verification](https://mlsanity.com/paper/1904.04589/) (2019)
- [Golden Gemini is All You Need: Finding the Sweet Spots for Speaker Verification](https://mlsanity.com/paper/2312.03620/) (2023)
- [SVDD 2024: The Inaugural Singing Voice Deepfake Detection Challenge](https://mlsanity.com/paper/2408.16132/) (2024)

## Built on by (most cited first, 7 of 7)

- [Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing](https://mlsanity.com/paper/2504.05657/) (2025)
- [Towards Quantifying and Reducing Language Mismatch Effects in Cross-Lingual Speech Anti-Spoofing](https://mlsanity.com/paper/2409.08346/) (2024)
- [Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learning](https://mlsanity.com/paper/2601.02983/) (2026)
- [AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan](https://mlsanity.com/paper/2604.08184/) (2026)
- [Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalization](https://mlsanity.com/paper/2502.12672/) (2025)
- [Quantizer-Aware Hierarchical Neural Codec Modeling for Speech Deepfake Detection](https://mlsanity.com/paper/2603.16914/) (2026)
- [Supervised Post-training of Speech Foundation Models for Robust Adaptation in Speech Deepfake Detection](https://mlsanity.com/paper/2606.25328/) (2026)

## Abstract (arXiv)

This work details our approach to achieving a leading system with a 1.79% pooled equal error rate (EER) on the evaluation set of the Controlled Singing Voice Deepfake Detection (CtrSVDD). The rapid advancement of generative AI models presents significant challenges for detecting AI-generated deepfake singing voices, attracting increased research attention. The Singing Voice Deepfake Detection (SVDD) Challenge 2024 aims to address this complex task. In this work, we explore the ensemble methods, utilizing speech foundation models to develop robust singing voice anti-spoofing systems. We also introduce a novel Squeeze-and-Excitation Aggregation (SEA) method, which efficiently and effectively integrates representation features from the speech foundation models, surpassing the performance of our other individual systems. Evaluation results confirm the efficacy of our approach in detecting deepfake singing voices. The codes can be accessed at https://github.com/Anmol2059/SVDD2024.