# DeepAgent: A Dual Stream Multi Agent Fusion for Robust Multimodal Deepfake Detection

arXiv: https://arxiv.org/abs/2512.07351 · HTML: https://mlsanity.com/paper/2512.07351/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection](https://mlsanity.com/paper/2408.09647/) (2024)
- [Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes](https://mlsanity.com/paper/2401.15668/) (2024)
- [DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection](https://mlsanity.com/paper/2306.00863/) (2023)
- [Zero-Shot Fake Video Detection by Audio-Visual Consistency](https://mlsanity.com/paper/2406.07854/) (2024)
- [Real Face Foundation Representation Learning for Generalized Deepfake Detection](https://mlsanity.com/paper/2303.08439/) (2023)
- [A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features](https://mlsanity.com/paper/2208.00788/) (2022)
- [CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation](https://mlsanity.com/paper/2505.15233/) (2025)
- [Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights](https://mlsanity.com/paper/2411.07650/) (2024)
- [Adaptive Meta-Learning for Robust Deepfake Detection: A Multi-Agent Framework to Data Drift and Model Generalization](https://mlsanity.com/paper/2411.08148/) (2024)
- [GazeForensics: DeepFake Detection via Gaze-guided Spatial Inconsistency Learning](https://mlsanity.com/paper/2311.07075/) (2023)
- [Adaptive Ensemble Learning: Boosting Model Performance through Intelligent Feature Fusion in Deep Neural Networks](https://mlsanity.com/paper/2304.02653/) (2023)
- [Self-DenseMobileNet: A Robust Framework for Lung Nodule Classification using Self-ONN and Stacking-based Meta-Classifier](https://mlsanity.com/paper/2410.12584/) (2024)
- [Unsupervised Cross-Modal Audio Representation Learning from Unstructured Multilingual Text](https://mlsanity.com/paper/2003.12265/) (2020)

## Built on by (most cited first, 1 of 1)

- [Detecting AI-Generated Video: A Vision-Language Dual-View Survey](https://mlsanity.com/paper/2607.10787/) (2026)

## Abstract (arXiv)

The increasing use of synthetic media, particularly deepfakes, is an emerging challenge for digital content verification. Although recent studies use both audio and visual information, most integrate these cues within a single model, which remains vulnerable to modality mismatches, noise, and manipulation. To address this gap, we propose DeepAgent, an advanced multi-agent collaboration framework that simultaneously incorporates both visual and audio modalities for the effective detection of deepfakes. DeepAgent consists of two complementary agents. Agent-1 examines each video with a streamlined AlexNet-based CNN to identify the symbols of deepfake manipulation, while Agent-2 detects audio-visual inconsistencies by combining acoustic features, audio transcriptions from Whisper, and frame-reading sequences of images through EasyOCR. Their decisions are fused through a Random Forest meta-classifier that improves final performance by taking advantage of the different decision boundaries learned by each agent. This study evaluates the proposed framework using three benchmark datasets to demonstrate both component-level and fused performance. Agent-1 achieves a test accuracy of 94.35% on the combined Celeb-DF and FakeAVCeleb datasets. On the FakeAVCeleb dataset, Agent-2 and the final meta-classifier attain accuracies of 93.69% and 81.56%, respectively. In addition, cross-dataset validation on DeepFakeTIMIT confirms the robustness of the meta-classifier, which achieves a final accuracy of 97.49%, and indicates a strong capability across diverse datasets. These findings confirm that hierarchy-based fusion enhances robustness by mitigating the weaknesses of individual modalities and demonstrate the effectiveness of a multi-agent approach in addressing diverse types of manipulations in deepfakes.