# Deepfake Audio Detection Using Self-supervised Fusion Representations

arXiv: https://arxiv.org/abs/2605.03420 · HTML: https://mlsanity.com/paper/2605.03420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 5 May 2026.

## Builds on (most cited first, 18 of 18)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech](https://mlsanity.com/paper/2102.05889/) (2021)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [SpoofCeleb: Speech Deepfake Detection and SASV In The Wild](https://mlsanity.com/paper/2409.17285/) (2024)
- [ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan](https://mlsanity.com/paper/2109.00535/) (2021)
- [Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0](https://mlsanity.com/paper/2409.11909/) (2024)
- [WavLM model ensemble for audio deepfake detection](https://mlsanity.com/paper/2408.07414/) (2024)
- [Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models](https://mlsanity.com/paper/2405.02179/) (2024)
- [ESDD2: Environment-Aware Speech and Sound Deepfake Detection Challenge Evaluation Plan](https://mlsanity.com/paper/2601.07303/) (2026)

## Abstract (arXiv)

This paper describes a submission to the Environment-Aware Speech and Sound Deepfake Detection Challenge (ESDD2) 2026, which addresses component-level deepfake detection using the CompSpoofV2 dataset, where speech and environmental sounds may be independently manipulated. To address this challenge, a dual-branch deepfake detection framework is proposed to jointly model speech and environmental contextual representations from input audio. Two pretrained models, XLS-R for speech and BEATs for environmental sound, are used to extract complementary contextual representations. A Matching Head is introduced to model representation differences through statistical normalization and representation interaction, enabling estimation of the original class. In parallel, multi-head cross-attention enables effective information exchange between speech and environmental components. The refined representations are processed with residual connections and layer normalization, and passed to an AASIST classifier to predict speech-based and environment-based spoofing probabilities. The model outputs original, speech, and environment predictions. On the test set, the proposed system achieves an F1-score of 70.20% and an environmental EER of 16.54%, outperforming the baseline system.