# Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learning

arXiv: https://arxiv.org/abs/2601.02983 · HTML: https://mlsanity.com/paper/2601.02983/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 30 of 34)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model](https://mlsanity.com/paper/2503.05132/) (2025)
- [Prompting Large Language Models with Speech Recognition Abilities](https://mlsanity.com/paper/2307.11795/) (2023)
- [Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models](https://mlsanity.com/paper/2503.02318/) (2025)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection](https://mlsanity.com/paper/2107.12710/) (2021)
- [Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning](https://mlsanity.com/paper/2506.04207/) (2025)
- [Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start](https://mlsanity.com/paper/2505.22334/) (2025)
- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation](https://mlsanity.com/paper/2401.13527/) (2024)
- [Attentive Merging of Hidden Embeddings from Pre-trained Speech Model for Anti-spoofing Detection](https://mlsanity.com/paper/2406.10283/) (2024)
- [ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection](https://mlsanity.com/paper/2505.11079/) (2025)
- [CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection](https://mlsanity.com/paper/2406.02438/) (2024)
- [Heterogeneity over Homogeneity: Investigating Multilingual Speech Pre-Trained Models for Detecting Audio Deepfake](https://mlsanity.com/paper/2404.00809/) (2024)
- [FakeMusicCaps: a Dataset for Detection and Attribution of Synthetic Music Generated via Text-to-Music Models](https://mlsanity.com/paper/2409.10684/) (2024)
- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [Comprehensive Layer-wise Analysis of SSL Models for Audio Deepfake Detection](https://mlsanity.com/paper/2502.03559/) (2025)
- [QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions](https://mlsanity.com/paper/2503.20290/) (2025)
- [Defense Against Adversarial Attacks on Audio DeepFake Detection](https://mlsanity.com/paper/2212.14597/) (2022)
- [FSD: An Initial Chinese Dataset for Fake Song Detection](https://mlsanity.com/paper/2309.02232/) (2023)
- [From Audio Deepfake Detection to AI-Generated Music Detection -- A Pathway and Overview](https://mlsanity.com/paper/2412.00571/) (2024)
- [EnvSDD: Benchmarking Environmental Sound Deepfake Detection](https://mlsanity.com/paper/2505.19203/) (2025)
- [Singing Voice Graph Modeling for SingFake Detection](https://mlsanity.com/paper/2406.03111/) (2024)
- [SVDD 2024: The Inaugural Singing Voice Deepfake Detection Challenge](https://mlsanity.com/paper/2408.16132/) (2024)
- [Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024](https://mlsanity.com/paper/2409.02302/) (2024)

## Built on by (most cited first, 6 of 6)

- [Towards Explicit Acoustic Evidence Perception in Audio LLMs for Speech Deepfake Detection](https://mlsanity.com/paper/2601.23066/) (2026)
- [Dual-Branch Gated Fusion for Open-Set Audio Deepfake Source Tracing](https://mlsanity.com/paper/2606.10223/) (2026)
- [APEX: Audio Prototype EXplanations for Classification Tasks](https://mlsanity.com/paper/2605.10153/) (2026)
- [XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models](https://mlsanity.com/paper/2606.16137/) (2026)
- [ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection](https://mlsanity.com/paper/2609.03620/) (2026)
- [SEAR: Spoofing Evidence-Grounded Audio Reasoning Benchmark for Audio Language Models](https://mlsanity.com/paper/2609.39847/) (2026)

## Abstract (arXiv)

Recent advances in audio large language models (ALLMs) have made high-quality synthetic audio widely accessible, increasing the risk of malicious audio deepfakes across speech, environmental sounds, singing voice, and music. Real-world audio deepfake detection (ADD) therefore requires all-type detectors that generalize across heterogeneous audio and provide interpretable decisions. Given the strong multi-task generalization ability of ALLMs, we first investigate their performance on all-type ADD under both supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). However, SFT using only binary real/fake labels tends to reduce the model to a black-box classifier, sacrificing interpretability. Meanwhile, vanilla RFT under sparse supervision is prone to reward hacking and can produce hallucinated, ungrounded rationales. To address this, we propose an automatic annotation and polishing pipeline that constructs Frequency-Time structured chain-of-thought (CoT) rationales, producing ~340K cold-start demonstrations. Building on CoT data, we propose Frequency Time-Group Relative Policy Optimization (FT-GRPO), a two-stage training paradigm that cold-starts ALLMs with SFT and then applies GRPO under rule-based frequency-time constraints. Experiments demonstrate that FT-GRPO achieves state-of-the-art performance on all-type ADD while producing interpretable, FT-grounded rationales. The data and code are available online.