# Anomaly Detection and Localization for Speech Deepfakes via Feature Pyramid Matching

arXiv: https://arxiv.org/abs/2503.18032 · HTML: https://mlsanity.com/paper/2503.18032/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [MLAAD: The Multi-Language Audio Anti-Spoofing Dataset](https://mlsanity.com/paper/2401.09512/) (2024)
- [AI-Synthesized Voice Detection Using Neural Vocoder Artifacts](https://mlsanity.com/paper/2304.13085/) (2023)
- [Audio Deepfake Detection with Self-Supervised WavLM and Multi-Fusion Attentive Classifier](https://mlsanity.com/paper/2312.08089/) (2023)
- [Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders](https://mlsanity.com/paper/2210.10570/) (2022)
- [TIMIT-TTS: a Text-to-Speech Dataset for Multimodal Synthetic Media Detection](https://mlsanity.com/paper/2209.08000/) (2022)
- [Leveraging Mixture of Experts for Improved Speech Deepfake Detection](https://mlsanity.com/paper/2409.16077/) (2024)
- [One-Class Knowledge Distillation for Spoofing Speech Detection](https://mlsanity.com/paper/2309.08285/) (2023)
- [One-Class Learning with Adaptive Centroid Shift for Audio Deepfake Detection](https://mlsanity.com/paper/2406.16716/) (2024)
- [Open Challenges in Synthetic Speech Detection](https://mlsanity.com/paper/2209.07180/) (2022)
- [All-for-One and One-For-All: Deep learning-based feature fusion for Synthetic Speech Detection](https://mlsanity.com/paper/2307.15555/) (2023)
- [Comparative Analysis of ASR Methods for Speech Deepfake Detection](https://mlsanity.com/paper/2411.17349/) (2024)

## Built on by (most cited first, 4 of 4)

- [Attention-based Mixture of Experts for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2509.17585/) (2025)
- [End-to-end Audio Deepfake Detection from RAW Waveforms: a RawNet-Based Approach with Cross-Dataset Evaluation](https://mlsanity.com/paper/2504.20923/) (2025)
- [Acoustic and perceptual differences between standard and accented speech and their voice clones](https://mlsanity.com/paper/2604.01562/) (2026)
- [Explainable-by-Design Audio Deepfake Detection via Wiener-Hopf Linear Prediction](https://mlsanity.com/paper/2607.12584/) (2026)

## Abstract (arXiv)

The rise of AI-driven generative models has enabled the creation of highly realistic speech deepfakes - synthetic audio signals that can imitate target speakers' voices - raising critical security concerns. Existing methods for detecting speech deepfakes primarily rely on supervised learning, which suffers from two critical limitations: limited generalization to unseen synthesis techniques and a lack of explainability. In this paper, we address these issues by introducing a novel interpretable one-class detection framework, which reframes speech deepfake detection as an anomaly detection task. Our model is trained exclusively on real speech to characterize its distribution, enabling the classification of out-of-distribution samples as synthetically generated. Additionally, our framework produces interpretable anomaly maps during inference, highlighting anomalous regions across both time and frequency domains. This is done through a Student-Teacher Feature Pyramid Matching system, enhanced with Discrepancy Scaling to improve generalization capabilities across unseen data distributions. Extensive evaluations demonstrate the superior performance of our approach compared to the considered baselines, validating the effectiveness of framing speech deepfake detection as an anomaly detection problem.