JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. Existing adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes prompt embeddings toward authenticity-affirming semantics under a token-proximity constraint. Experiments on forensic VLM benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 40 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
34 of 40 references.
Closest ancestors · 9
Same direction, newest first
- 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model5 Dec
- 2024ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization14 Oct
- 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models3 Oct
- 2024Probing the Robustness of Vision-Language Pretrained Models: A Multimodal Adversarial Attack Approach24 Aug
- 2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant19 Aug
- 2024Adversarial Robustness for Visual Grounding of Multimodal Large Language Models16 May
- 2024Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective30 Apr
- 2024Improving Adversarial Transferability of Vision-Language Pre-training Models through Collaborative Multimodal Interaction16 Mar
- 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models14 Mar
Other references · 25
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
- 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
- 2022Diffusion Models for Adversarial Purificationcited by 469
- 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
- 2021M2TR: Multi-modal Multi-scale Transformers for Deepfake Detectioncited by 114
- 2022Protecting Celebrities from DeepFake with Identity Consistency Transformercited by 95
13 more
- 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
- 2017Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detectioncited by 56
- 2021OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wildcited by 50
- 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 47
- 2023Improving Interpretation Faithfulness for Vision Transformerscited by 12
- 2024XAI-Based Detection of Adversarial Attacks on Deepfake Detectorscited by 9
- 2025ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attackcited by 6
- 2026Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoningcited by 4
- 2026Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbationscited by 3
- 2025DF-LLaVA: Unlocking MLLMs for Synthetic Image Detection via Knowledge Injection and Conflict-Driven Self-Reflectioncited by 2
- 2026SARA: Stress Test Reasoning in Audio Deepfake Detectioncited by 2
- 2023Evading Detection Actively: Toward Anti-Forensics against Forgery Localizationcited by 1
- 2026Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidancecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models
- 2025FORGE: Forensic Reasoning with Grounded Evidence
- 2025REVEAL -- Reasoning and Evaluation of Visual Evidence through Aligned Language
- 2026DE-FIVE: Detecting Malicious Image Prompts via Fourier Features and Image Vector Embeddings
- 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
- 2026Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
- 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt
- 2026Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations