Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities
Not on the map. Why?
Abstract (arXiv)
As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex, multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench benchmark, which integrates diverse multimodal forgery detection algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal multimodal defense.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
41 of 60 references.
References · 41
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2019Invariant Risk Minimizationcited by 1,854
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
29 more
- 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
- 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
- 2020Deep Domain-Adversarial Image Generation for Domain Generalisationcited by 231
- 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
- 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 194
- 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 181
- 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 118
- 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 92
- 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
- 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
- 2024UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them Allcited by 49
- 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
- 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
- 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 24
- 2024Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Contentcited by 22
- 2025Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detectioncited by 21
- 2025Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensicscited by 21
- 2025Continual Multimodal Contrastive Learningcited by 20
- 2025Revisiting Multimodal Positional Encoding in Vision-Language Modelscited by 16
- 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
- 2025OmniVec2 -- A Novel Transformer based Network for Large Scale Multimodal and Multitask Learningcited by 14
- 2024Open-Vocabulary Segmentation with Unpaired Mask-Text Supervisioncited by 12
- 2024Towards Modality Generalization: A Benchmark and Prospective Analysiscited by 10
- 2025Scaling Up AI-Generated Image Detection with Generator-Aware Prototypescited by 9
- 2025AV-Deepfake1M++: A Large-Scale Audio-Visual Deepfake Benchmark with Real-World Perturbationscited by 8
- 2025Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalizationcited by 6
- 2025KLASSify to Verify: Audio-Visual Deepfake Detection Using SSL-based Audio and Handcrafted Visual Featurescited by 5
- 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026SRENet: Spectral Re-Entry Network for Point Cloud Action Recognitioncited by 1
- 2026LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detectioncited by 1
- 2026A Heuristic Perspective on Debiasing Language Modelscited by 1
- 2026PhenoYieldNet: Learning Crop-Aware Phenological Responses for Multi-Crop Yield Predictioncited by 0
- 2026Do correct answers have stable latent knowledge support?cited by 0
- 2026SuperAI: Merging Capabilities Across Heterogeneous Foundation Modelscited by 0
- 2026SuperAI: Merging Capabilities Across Heterogeneous Foundation Models11 Oct 2026
- 2026Do correct answers have stable latent knowledge support?11 Oct 2026
- 2026A Heuristic Perspective on Debiasing Language Models1 Aug 2026
- 2026LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection28 Jul 2026
- 2026SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition2 Jun 2026
- 2026PhenoYieldNet: Learning Crop-Aware Phenological Responses for Multi-Crop Yield Prediction22 May 2026