Beyond Semantic Priors: Mitigating Optimization Collapse for Generalizable Visual Forensics
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
While Vision-Language Models (VLMs) like CLIP have emerged as a dominant paradigm for generalizable deepfake detection, a representational disconnect remains: their semantic-centric pre-training is ill-suited for capturing non-semantic artifacts inherent to hyper-realistic synthesis. In this work, we identify a failure mode termed Optimization Collapse, where detectors trained with Sharpness-Aware Minimization (SAM) degenerate to random guessing on non-semantic forgeries once the perturbation radius exceeds a narrow threshold. To theoretically formalize this collapse, we propose the Critical Optimization Radius (COR) to quantify the geometric stability of the optimization landscape, and leverage the Gradient Signal-to-Noise Ratio (GSNR) to measure generalization potential. We establish a theorem proving that COR increases monotonically with GSNR, thereby revealing that the geometric instability of SAM optimization originates from degraded intrinsic generalization potential. This result identifies the layer-wise attenuation of GSNR as the root cause of Optimization Collapse in detecting non-semantic forgeries. Although naively reducing perturbation radius yields stable convergence under SAM, it merely treats the symptom without mitigating the intrinsic generalization degradation, necessitating enhanced gradient fidelity. Building on this insight, we propose the Contrastive Regional Injection Transformer (CoRIT), which integrates a computationally efficient Contrastive Gradient Proxy (CGP) with three training-free strategies: Region Refinement Mask to suppress CGP variance, Regional Signal Injection to preserve CGP magnitude, and Hierarchical Representation Integration to attain more generalizable representations. Extensive experiments demonstrate that CoRIT mitigates optimization collapse and achieves state-of-the-art generalization across cross-domain and universal forgery benchmarks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 1 of its 89 references that are in the map. First posted on arXiv 25 Mar 2026.
Builds on
1 of 89 references.
References · 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Specialist-Generalist Fusion with Outcome-Supervised Rationales for Deepfake Detection
- 2026Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models
- 2026Cross-Domain Generalization Limits of Vision Foundation Models in Facial Deepfake Detection
- 2026ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models
- 2026Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
- 2025FORGE: Forensic Reasoning with Grounded Evidence
- 2025AuthGuard: Generalizable Deepfake Detection via Language Guidance
- 2025Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning