EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks. Existing defense methods predominantly focus on sample-level defense, which relies on the knowledge of training data or triggers. However, identifying whether a given model is backdoored remains a critical but unexplored task. To fill this gap, we propose EntropyScan, a lightweight and trigger-agnostic method for model-level backdoor detection in LVLMs. We first observe that backdoor injection disrupts the cross-modal alignment, resulting in pronounced structural anomalies in visual attention allocation on benign samples. Based on this insight, EntropyScan detects the backdoor models by quantifying such attention deviations. Specifically, it extracts visual attention distributions from the initial layers of the Large Language Model (LLM) and applies Tsallis entropy to capture these structural distortions. By employing a reference-anchored Z-score normalization on a small set of benign samples, it effectively identifies the backdoored model. Extensive experiments across two LVLMs architectures and three advanced attack scenarios show that EntropyScan achieves an F1 score of 98.5% in average and an AUC of 96.6%. Our code will be publicly available soon.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 57 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
15 of 57 references.
Closest ancestors · 5
Same direction, newest first
- 2026TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuning29 Jan
- 2025Backdoor Attack on Vision Language Models with Stealthy Semantic Manipulation8 Jun
- 2025Robust Anti-Backdoor Instruction Tuning in LVLMs4 Jun
- 2025Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving9 May
- 2024Test-Time Backdoor Attacks on Multimodal Large Language Models13 Feb
Other references · 10
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
- 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
- 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
- 2025Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
- 2026Test-Time Attention Purification for Backdoored Large Vision Language Models
- 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift
- 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuning
- 2026TMT: Runtime Backdoor Detection for Vision-Language-Action Policies on Unseen Tasks
- 2026Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis
- 2026A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models
- 2025TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models