# What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection

arXiv: https://arxiv.org/abs/2609.33375 · HTML: https://mlsanity.com/paper/2609.33375/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?](https://mlsanity.com/paper/2106.12914/) (2021)
- [The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio](https://mlsanity.com/paper/2405.04880/) (2024)
- [Towards generalisable and calibrated synthetic speech detection with self-supervised representations](https://mlsanity.com/paper/2309.05384/) (2023)
- [Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0](https://mlsanity.com/paper/2409.11909/) (2024)
- [Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing](https://mlsanity.com/paper/2504.05657/) (2025)
- [CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems](https://mlsanity.com/paper/2406.07237/) (2024)
- [Cross-Domain Audio Deepfake Detection: Dataset and Analysis](https://mlsanity.com/paper/2404.04904/) (2024)
- [Generalization of Spoofing Countermeasures: a Case Study with ASVspoof 2015 and BTAS 2016 Corpora](https://mlsanity.com/paper/1901.08025/) (2019)
- [CompSpoof: A Dataset and Joint Learning Framework for Component-Level Audio Anti-spoofing Countermeasures](https://mlsanity.com/paper/2509.15804/) (2025)
- [Listening Between the Lines: Synthetic Speech Detection Disregarding Verbal Content](https://mlsanity.com/paper/2402.05567/) (2024)
- [Interpretable Temporal Class Activation Representation for Audio Spoofing Detection](https://mlsanity.com/paper/2406.08825/) (2024)

## Abstract (arXiv)

The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes. We therefore compare 12 acoustic representations using a shared low-capacity linear classifier to identify effective evidence under codec shift. The analysis shows that hierarchical XLS-R leads on the pooled test set, while pooled no-vocals residual statistics perform best on the unseen-codec condition, revealing complementary behavior across generation conditions. Building on this finding, we propose MN-P, a dual-view detector that integrates an utterance-level pooled no-vocals representation with token-level XLS-R features through adaptive gating. The proposed MN-P reduces EER by 54.2% overall and by 60.9% on the codec-unseen condition relative to the best-performing retrained state-of-the-art system, with consistent gains across different detector backends. These results indicate that pooled no-vocals residual statistics provide effective complementary evidence for cross-generation speech deepfake detection.