SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
Deepfake (DF) audio detectors still struggle to generalize to out of distribution inputs. A central reason is spectral bias, the tendency of neural networks to learn low-frequency structure before high-frequency (HF) details, which both causes DF generators to leave HF artifacts and leaves those same artifacts under-exploited by common detectors. To address this gap, we propose Spectral-cONtrastive Audio Residuals (SONAR), a frequency-guided framework that explicitly disentangles an audio signal into complementary representations. An XLSR encoder captures the dominant low-frequency content, while the same cloned path, preceded by learnable SRM, value-constrained high-pass filters, distills faint HF residuals. Frequency cross-attention reunites the two views for long- and short-range frequency dependencies, and a frequency-aware Jensen-Shannon contrastive loss pulls real content-noise pairs together while pushing fake embeddings apart, accelerating optimization and sharpening decision boundaries. Evaluated on the ASVspoof 2021 and in-the-wild benchmarks, SONAR attains state-of-the-art performance and converges four times faster than strong baselines. By elevating faint high-frequency residuals to first-class learning signals, SONAR unveils a fully data-driven, frequency-guided contrastive framework that splits the latent space into two disjoint manifolds: natural-HF for genuine audio and distorted-HF for synthetic audio, thereby sharpening decision boundaries. Because the scheme operates purely at the representation level, it is architecture-agnostic and, in future work, can be seamlessly integrated into any model or modality where subtle high-frequency cues are decisive.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 26 Nov 2025.
Builds on
42 of 61 references.
References · 42
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2018On the Spectral Bias of Neural Networkscited by 1,341
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
- 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 523
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
30 more
- 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
- 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 180
- 2019Towards Understanding the Spectral Bias of Deep Learningcited by 174
- 2020End-to-end anti-spoofing with RawNet2cited by 164
- 2020Two-branch Recurrent Network for Isolating Deepfakes in Videoscited by 161
- 2018Learning Rich Features for Image Manipulation Detectioncited by 154
- 2020Model-Based Deep Learningcited by 141
- 2017Convergence Analysis of Two-layer Neural Networks with ReLU Activationcited by 137
- 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 135
- 2022Does Audio Deepfake Detection Generalize?cited by 135
- 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 121
- 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 117
- 2022Overview frequency principle/spectral bias in deep learningcited by 69
- 2021RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofingcited by 67
- 2021BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detectioncited by 60
- 2021On the Frequency Bias of Generative Modelscited by 60
- 2021SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimizationcited by 59
- 2021ADD: Frequency Attention and Multi-View based Knowledge Distillation to Detect Low-Quality Compressed Deepfake Imagescited by 48
- 2021Investigating self-supervised front ends for speech spoofing countermeasurescited by 46
- 2021Spectral Bias in Practice: The Role of Function Frequency in Generalizationcited by 37
- 2024XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detectioncited by 30
- 2023Audio Deepfake Detection with Self-Supervised WavLM and Multi-Fusion Attentive Classifiercited by 27
- 2024RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detectioncited by 27
- 2024Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detectioncited by 25
- 2016Deep Edge Guided Recurrent Residual Learning for Image Super-Resolutioncited by 22
- 2022WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generationcited by 22
- 2021ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plancited by 20
- 2024Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0cited by 20
- 2024A robust audio deepfake detection system via multi-view featurecited by 19
- 2022Deepfake audio detection by speaker verificationcited by 11
Built on by
Citing papers per year: 2026: 1.
1 paper.