Specialist-Generalist Fusion with Outcome-Supervised Rationales for Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
Generalizable deepfake detection requires complementary forensic and semantic visual evidence. Specialist encoders capture subtle manipulation traces but can overfit to source-specific statistics, whereas MLLMs provide broader visual-semantic representations but may overlook fine forensic artifacts. We propose a two-stage detector in which an MLLM directly fuses patch-level features from a frozen forensic encoder with those from its native vision encoder and produces the authenticity decision itself. This specialist--generalist alignment provides the main cross-domain performance gain. We subsequently introduce outcome-supervised rationale tuning. The model generates a free-form visual rationale before its decision and is optimized using only the binary authenticity label and a format constraint, without task-specific rationale annotations. Rationale generation is optional at inference, so the tuned model can still return a direct binary score. On DF40, specialist--generalist alignment improves average cross-domain AUC from 89.85 to 93.32±0.44. Across six paired runs, rationale tuning obtains 93.50±0.42 and improves five of six paired models; the mean difference is small and not statistically conclusive. Results on SID-Set and legacy benchmarks, together with fusion, output-order, and continued-training controls, demonstrate the value of complementary visual representations and show that label-only rationale tuning can add an optional explanation mode while approximately preserving direct detection performance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
36 of 52 references.
References · 36
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
- 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
- 2019Face X-ray for More General Face Forgery Detectioncited by 469
- 2022Detecting Deepfakes with Self-Blended Imagescited by 332
24 more
- 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
- 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 241
- 2020Learning Self-Consistency for Deepfake Detectioncited by 179
- 2023DeepfakeBench: A Comprehensive Benchmark of Deepfake Detectioncited by 162
- 2020Two-branch Recurrent Network for Isolating Deepfakes in Videoscited by 161
- 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
- 2020The DeepFake Detection Challenge (DFDC) Datasetcited by 125
- 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
- 2018Synthetic Depth-of-Field with a Single-Camera Mobile Phonecited by 70
- 2023AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectorscited by 64
- 2024Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuningcited by 57
- 2022SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakescited by 49
- 2021Beyond the Spectrum: Detecting Deepfakes via Re-Synthesiscited by 44
- 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detectorcited by 38
- 2025Ivy-Fake: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detectioncited by 34
- 2023Controllable Guide-Space for Generalizable Face Forgery Detectioncited by 21
- 2023Detecting Generated Images by Real Images Onlycited by 20
- 2025Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detectioncited by 20
- 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detectioncited by 8
- 2026EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detectioncited by 6
- 2025PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detectioncited by 5
- 2025Practical Manipulation Model for Robust Deepfake Detectioncited by 4
- 2026Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detectioncited by 2
- 2026Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detectioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025AuthGuard: Generalizable Deepfake Detection via Language Guidance
- 2026Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection
- 2026Beyond Semantic Priors: Mitigating Optimization Collapse for Generalizable Visual Forensics
- 2025Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning
- 2025FORGE: Forensic Reasoning with Grounded Evidence
- 2026Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
- 2026LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection
- 2026VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection