# Listening Deepfake Detection: A New Perspective Beyond Speaking-Centric Forgery Analysis

arXiv: https://arxiv.org/abs/2604.12650 · HTML: https://mlsanity.com/paper/2604.12650/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion](https://mlsanity.com/paper/2204.08451/) (2022)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [Deepfake Video Detection Using Convolutional Vision Transformer](https://mlsanity.com/paper/2102.11126/) (2021)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [ADD 2023: the Second Audio Deepfake Detection Challenge](https://mlsanity.com/paper/2305.13774/) (2023)
- [Sharp Multiple Instance Learning for DeepFake Video Detection](https://mlsanity.com/paper/2008.04585/) (2020)
- [VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior](https://mlsanity.com/paper/2312.01841/) (2023)
- [Video Generative Adversarial Networks: A Review](https://mlsanity.com/paper/2011.02250/) (2020)
- [Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detection](https://mlsanity.com/paper/2401.05746/) (2024)
- [SafeEar: Content Privacy-Preserving Audio Deepfake Detection](https://mlsanity.com/paper/2409.09272/) (2024)
- [ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan](https://mlsanity.com/paper/2109.00535/) (2021)
- [GM-DF: Generalized Multi-Scenario Deepfake Detection](https://mlsanity.com/paper/2406.20078/) (2024)
- [Interactive Generative Adversarial Networks for Facial Expression Generation in Dyadic Interactions](https://mlsanity.com/paper/1801.09092/) (2018)
- [Identity-Driven Multimedia Forgery Detection via Reference Assistance](https://mlsanity.com/paper/2401.11764/) (2024)
- [Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces](https://mlsanity.com/paper/2401.13516/) (2024)
- [Perceptual Conversational Head Generation with Regularized Driver and Enhanced Renderer](https://mlsanity.com/paper/2206.12837/) (2022)
- [Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations](https://mlsanity.com/paper/2503.22121/) (2025)

## Abstract (arXiv)

Existing deepfake detection research has primarily focused on scenarios where the manipulated subject is actively speaking, i.e., generating fabricated content by altering the speaker's appearance or voice. However, in realistic interaction settings, attackers often alternate between falsifying speaking and listening states to mislead their targets, thereby enhancing the realism and persuasiveness of the scenario. Although the detection of 'listening deepfakes' remains largely unexplored and is hindered by a scarcity of both datasets and methodologies, the relatively limited quality of synthesized listening reactions presents an excellent breakthrough opportunity for current deepfake detection efforts. In this paper, we present the task of Listening Deepfake Detection (LDD). We introduce ListenForge, the first dataset specifically designed for this task, constructed using five Listening Head Generation (LHG) methods. To address the distinctive characteristics of listening forgeries, we propose MANet, a Motion-aware and Audio-guided Network that captures subtle motion inconsistencies in listener videos while leveraging speaker's audio semantics to guide cross-modal fusion. Extensive experiments demonstrate that existing Speaking Deepfake Detection (SDD) models perform poorly in listening scenarios. In contrast, MANet achieves significantly superior performance on ListenForge. Our work highlights the necessity of rethinking deepfake detection beyond the traditional speaking-centric paradigm and opens new directions for multimodal forgery analysis in interactive communication settings. The dataset and code are available at https://anonymous.4open.science/r/LDD-B4CB.