mlsanityData to 7 Oct 2026

FreqDoor: A Hidden Trojan in the Frequency Domain for Backdoor Attacks on Vision-Language Models

arXiv 2609.07048 ↗First posted 7 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Vision-language models (VLMs) have recently shown excellent progress in open-ended image-to-text generation. However, their multimodal nature makes them persistently vulnerable to backdoor attacks. Existing backdoor triggers for VLMs are either spatial, textual, or bimodal, which may yield localized or recognizable trigger patterns. In this work, we explore a different attack surface and propose textsc FreqDoor, a training-time backdoor attack that implants triggers in the frequency domain. textsc FreqDoor mixes amplitude-spectrum components from a trigger-source image selectively while preserving the phase of a clean image to generate a spatially distributed and visually imperceptible trigger without modifying the textual input. We evaluate the attack on BLIP-2, InstructBLIP, and LLaVA for image captioning and visual question answering. On Flickr8k, textsc FreqDoor achieves attack success rates of 99.6%, 99.8%, and 98.4% on the three models, respectively, while preserving the semantic quality of the generated captions. On VQAv2, the corresponding attack success rates are 99.6%, 92.4%, and 79.6%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

37 of 45 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models24 Nov
  2. 2025VisualDAN: Exposing Vulnerabilities in VLMs with Visual-Driven DAN Commands9 Oct
  3. 2025Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models25 Feb
  4. 2025Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving23 Jan
  5. 2024Backdooring Vision-Language Models with Out-Of-Distribution Data2 Oct
  6. 2024TrojVLM: Backdoor Attack Against Vision Language Models28 Sep
  7. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends10 Jul
  8. 2024Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Models16 Apr
  9. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
  10. 2024Test-Time Backdoor Attacks on Multimodal Large Language Models13 Feb
  11. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb
  12. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  10. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  11. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  12. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
13 more
  1. 2022Diffusion Models for Adversarial Purificationcited by 469
  2. 2018Spectral Signatures in Backdoor Attackscited by 457
  3. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  4. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  5. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  6. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  7. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learningcited by 120
  8. 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 62
  9. 2023Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Triggercited by 40
  10. 2021Dual-Key Multimodal Backdoors for Visual Question Answeringcited by 35
  11. 2025BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Modelscited by 15
  12. 2022Imperceptible and Multi-channel Backdoor Attack against Deep Neural Networkscited by 10
  13. 2024Low-Frequency Black-Box Backdoor Attack via Evolutionary Algorithmcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Models
  2. 2025Concept-Guided Backdoor Attack on Vision Language Models
  3. 2026Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs
  4. 2025Backdoor Attack on Vision Language Models with Stealthy Semantic Manipulation
  5. 2025MTAttack: Multi-Target Backdoor Attacks against Large Vision-Language Models
  6. 2026Phantasia: Context-Adaptive Backdoors in Vision Language Models
  7. 2024Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIP
  8. 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift