# CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems

arXiv: https://arxiv.org/abs/2406.07237 · HTML: https://mlsanity.com/paper/2406.07237/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2024.

## Built on by (most cited first, 16 of 16)

- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment](https://mlsanity.com/paper/2507.02768/) (2025)
- [DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset](https://mlsanity.com/paper/2409.08731/) (2024)
- [Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024](https://mlsanity.com/paper/2409.02302/) (2024)
- [Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems](https://mlsanity.com/paper/2509.09204/) (2025)
- [AUDDT: A Unified Benchmark Toolkit for Audio and Speech Deepfake Detectors](https://mlsanity.com/paper/2509.21597/) (2025)
- [Alethia: A Foundational Encoder for Voice Deepfakes](https://mlsanity.com/paper/2605.00251/) (2026)
- [Ethical and Technical Limits of Deepfake Speech Datasets](https://mlsanity.com/paper/2606.10911/) (2026)
- [Towards Generalized Source Tracing for Codec-Based Deepfake Speech](https://mlsanity.com/paper/2506.07294/) (2025)
- [Towards Reliable Audio Deepfake Attribution and Model Recognition: A Multi-Level Autoencoder-Based Framework](https://mlsanity.com/paper/2508.02521/) (2025)
- [How to Label Resynthesized Audio: The Dual Role of Neural Audio Codecs in Audio Deepfake Detection](https://mlsanity.com/paper/2602.16343/) (2026)
- [What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection](https://mlsanity.com/paper/2603.14033/) (2026)
- [Quantizer-Aware Hierarchical Neural Codec Modeling for Speech Deepfake Detection](https://mlsanity.com/paper/2603.16914/) (2026)
- [What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection](https://mlsanity.com/paper/2609.33375/) (2026)
- [SEAR: Spoofing Evidence-Grounded Audio Reasoning Benchmark for Audio Language Models](https://mlsanity.com/paper/2609.39847/) (2026)
- [Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2610.16049/) (2026)

## Abstract (arXiv)

Current state-of-the-art (SOTA) codec-based audio synthesis systems can mimic anyone's voice with just a 3-second sample from that specific unseen speaker. Unfortunately, malicious attackers may exploit these technologies, causing misuse and security issues. Anti-spoofing models have been developed to detect fake speech. However, the open question of whether current SOTA anti-spoofing models can effectively counter deepfake audios from codec-based speech synthesis systems remains unanswered. In this paper, we curate an extensive collection of contemporary SOTA codec models, employing them to re-create synthesized speech. This endeavor leads to the creation of CodecFake, the first codec-based deepfake audio dataset. Additionally, we verify that anti-spoofing models trained on commonly used datasets cannot detect synthesized speech from current codec-based speech generation systems. The proposed CodecFake dataset empowers these models to counter this challenge effectively.