# Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems

arXiv: https://arxiv.org/abs/2509.09204 · HTML: https://mlsanity.com/paper/2509.09204/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [MLAAD: The Multi-Language Audio Anti-Spoofing Dataset](https://mlsanity.com/paper/2401.09512/) (2024)
- [Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection](https://mlsanity.com/paper/2406.17376/) (2024)
- [CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems](https://mlsanity.com/paper/2406.07237/) (2024)
- [LlamaPartialSpoof: An LLM-Driven Fake Speech Dataset Simulating Disinformation Generation](https://mlsanity.com/paper/2409.14743/) (2024)
- [EmoFake: An Initial Dataset for Emotion Fake Audio Detection](https://mlsanity.com/paper/2211.05363/) (2022)
- [Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing](https://mlsanity.com/paper/2305.19953/) (2023)
- [Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems](https://mlsanity.com/paper/2407.03645/) (2024)
- [Data Quality as Predictor of Voice Anti-Spoofing Generalization](https://mlsanity.com/paper/2103.14602/) (2021)
- [Range-Based Equal Error Rate for Spoof Localization](https://mlsanity.com/paper/2305.17739/) (2023)
- [A Practical Guide to Logical Access Voice Presentation Attack Detection](https://mlsanity.com/paper/2201.03321/) (2022)
- [Beyond Silence: Bias Analysis through Loss and Asymmetric Approach in Audio Anti-Spoofing](https://mlsanity.com/paper/2406.17246/) (2024)

## Built on by (most cited first, 3 of 3)

- [Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2509.20682/) (2025)
- [QAMO: Quality-aware Multi-centroid One-class Learning For Speech Deepfake Detection](https://mlsanity.com/paper/2509.20679/) (2025)
- [Teffic-Audio: Tell Fact from Fiction](https://mlsanity.com/paper/2607.28351/) (2026)

## Abstract (arXiv)

Audio deepfake detection (ADD) models are commonly evaluated using datasets that combine multiple synthesizers, with performance reported as a single Equal Error Rate (EER). However, this approach disproportionately weights synthesizers with more samples, underrepresenting others and reducing the overall reliability of EER. Additionally, most ADD datasets lack diversity in bona fide speech, often featuring a single environment and speech style (e.g., clean read speech), limiting their ability to simulate real-world conditions. To address these challenges, we propose bona fide cross-testing, a novel evaluation framework that incorporates diverse bona fide datasets and aggregates EERs for more balanced assessments. Our approach improves robustness and interpretability compared to traditional evaluation methods. We benchmark over 150 synthesizers across nine bona fide speech types and release a new dataset to facilitate further research at https://github.com/cyaaronk/audio_deepfake_eval.