# The Affective Bridge: Preserving Speech Representations while Enhancing Deepfake Detection vian emotional Constraints

arXiv: https://arxiv.org/abs/2512.11241 · HTML: https://mlsanity.com/paper/2512.11241/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models.
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Placed in

- Speech and audio language models / [Emotional speech synthesis and recognition](https://mlsanity.com/speech/emotional-speech-synthesis-and-recognition/)

## Builds on (most cited first, 18 of 18)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing](https://mlsanity.com/paper/2110.07205/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues](https://mlsanity.com/paper/2003.06711/) (2020)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Deepfake audio detection by speaker verification](https://mlsanity.com/paper/2209.14098/) (2022)
- [EmoFake: An Initial Dataset for Emotion Fake Audio Detection](https://mlsanity.com/paper/2211.05363/) (2022)
- [ASR-based Features for Emotion Recognition: A Transfer Learning Approach](https://mlsanity.com/paper/1805.09197/) (2018)
- [Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features?](https://mlsanity.com/paper/2402.00340/) (2024)
- [Easy, Interpretable, Effective: openSMILE for voice deepfake detection](https://mlsanity.com/paper/2408.15775/) (2024)
- [Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities](https://mlsanity.com/paper/2502.06803/) (2025)
- [GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations](https://mlsanity.com/paper/2503.20919/) (2025)
- [Exploring Green AI for Audio Deepfake Detection](https://mlsanity.com/paper/2403.14290/) (2024)
- [Artificial Emotion: A Survey of Theories and Debates on Realising Emotion in Artificial Intelligence](https://mlsanity.com/paper/2508.10286/) (2025)

## Abstract (arXiv)

Speech deepfake detection (DFD) has benefited from diverse acoustic and semantic speech representations, many of which encode valuable speech information and are costly to train. Prior work has shown that affective cues improve DFD, yet existing approaches either fuse emotion with other task-specific features in complex pipelines or directly fine-tune representations toward DFD objectives, risking distortion of the original speech representations that support downstream tasks such as speaker verification (SV) or automatic speech recognition (ASR). We propose a simpler approach: fine-tuning speech encoders on emotion recognition alone-without any DFD supervision, and training a lightweight support vector machine (SVM) on the frozen emotion-tuned representations for DFD. This preserves the original representation capacity for downstream tasks such as SV and ASR, while emergently improving DFD performance. Crucially, we find that emotion is uniquely effective as this bridging task: replacing it with speaker identity even degrades DFD performance, demonstrating that the benefit stems from emotion's role as a natural bridge between speech representation and DFD. Experiments on FakeOrReal and In-the-Wild show accuracy improvements of up to 6% and 2% with corresponding EER reductions, while analysis on ASVspoof 2019 LA reveals dataset-specific speaker bias in the real-speech subset. Code is available at supplementary materials.