mlsanityData to 14 Oct 2026

Towards Generalized Source Tracing for Codec-Based Deepfake Speech

arXiv 2506.07294 ↗First posted 8 Jun 2025cs.SD cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent attempts at source tracing for codec-based deepfake speech (CodecFake), generated by neural audio codec-based speech generation (CoSG) models, have exhibited suboptimal performance. However, how to train source tracing models using simulated CoSG data while maintaining strong performance on real CoSG-generated audio remains an open challenge. In this paper, we show that models trained solely on codec-resynthesized data tend to overfit to non-speech regions and struggle to generalize to unseen content. To mitigate these challenges, we introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding. Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 55 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

45 of 55 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  5. 2022Masked Autoencoders that Listencited by 273
  6. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 180
  7. 2019ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detectioncited by 178
  8. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  9. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 121
  10. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  11. 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 117
  12. 2018Auxiliary Tasks in Multi-task Learningcited by 108
33 more
  1. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  2. 2022ADD 2022: the First Audio Deep Synthesis Detection Challengecited by 96
  3. 2024ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scalecited by 91
  4. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  5. 2021RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofingcited by 67
  6. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  7. 2024Towards audio language modeling -- an overviewcited by 40
  8. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  9. 2023Improved DeepFake Detection Using Whisper Featurescited by 29
  10. 2024A Survey on Speech Deepfake Detectioncited by 27
  11. 2024Codec-SUPERB: An In-Depth Analysis of Sound Codec Modelscited by 26
  12. 2024CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systemscited by 16
  13. 2025A Preliminary Exploration with GPT-4o Voice Modecited by 16
  14. 2024Source Tracing of Audio Deepfake Systemscited by 15
  15. 2022An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audiocited by 14
  16. 2023The Impact of Silence on Speech Anti-Spoofingcited by 13
  17. 2023The defender's perspective on automatic speaker verification: An overviewcited by 9
  18. 2021Adversarial Sample Detection for Speaker Verification by Neural Vocoderscited by 8
  19. 2024Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec modelscited by 7
  20. 2023Distinguishing Neural Speech Synthesis Models Through Fingerprints in Speech Waveformscited by 6
  21. 2025CodecFake+: Codec-Based Resynthesized Data as a Proxy for Detecting CodecFake Speechcited by 6
  22. 2025Codec-Based Deepfake Source Tracing via Neural Audio Codec Taxonomycited by 6
  23. 2024Genuine-Focused Learning using Mask AutoEncoder for Generalized Fake Audio Detectioncited by 5
  24. 2024Neural Codec-based Adversarial Sample Detection for Speaker Verificationcited by 5
  25. 2025Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Conditioncited by 5
  26. 2025Source Verification for Speech Deepfakescited by 5
  27. 2025Towards Explainable Spoofed Speech Attribution and Detection:a Probabilistic Approach for Characterizing Speech Synthesizer Componentscited by 4
  28. 2022Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detectioncited by 3
  29. 2022Source Tracing: Detecting Voice Spoofingcited by 3
  30. 2024ADD 2023: Towards Audio Deepfake Detection and Analysis in the Wildcited by 3
  31. 2022Multimodal Transformer Distillation for Audio-Visual Synchronizationcited by 2
  32. 2024Towards Out-of-Distribution Detection in Vocoder Recognition via Latent Feature Reconstructioncited by 2
  33. 2025Listen, Analyze, and Adapt to Learn New Attacks: An Exemplar-Free Class Incremental Learning Method for Audio Deepfake Source Tracingcited by 2

Built on by

Not cited yet.