Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models
On the map: Diffusion models → Robustness, privacy and adversarial aspects
also in 1 more
Abstract (arXiv)
Multimodal Diffusion Language Models (MDLMs) have recently emerged as a competitive alternative to their autoregressive counterparts. Yet their vulnerability to backdoor attacks remains largely unexplored. In this work, we show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs, enabling attackers to manipulate model behavior via specific triggers while maintaining normal performance on clean inputs. However, defense strategies effective to these models are yet to emerge. To bridge this gap, we introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification). DiSP is driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality. Building on this, we purify the poisoned dataset using the compromised model itself, then fine-tune the model on the purified data to recover it to a clean one. Given such a specific design, DiSP can remove backdoors without requiring any auxiliary models or clean reference data. Extensive experiments demonstrate that our approach effectively mitigates backdoor effects, reducing the attack success rate (ASR) from over 90% to typically under 5%, while maintaining model performance on benign tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 60 of its 66 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
60 of 66 references.
Closest ancestors · 2
Same direction, newest first
- 2023VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models12 Jun
- 2022How to Backdoor Diffusion Models?11 Dec
Other references · 58
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2023A Survey on Multimodal Large Language Modelscited by 898
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2025Large Language Diffusion Modelscited by 872
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
- 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
- 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
- 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
- 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
- 2021Poisoning and Backdooring Contrastive Learningcited by 149
- 2022Backdoor Defense via Decoupling the Training Processcited by 149
- 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
- 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
- 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
- 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
- 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 112
- 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
- 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 100
- 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
- 2024BadEdit: Backdooring large language models by model editingcited by 84
- 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Modelscited by 83
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
- 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
- 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
- 2023Black-box Backdoor Defense via Zero-shot Image Purificationcited by 44
- 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
- 2022Data Poisoning Attacks Against Multimodal Encoderscited by 41
- 2024TrojVLM: Backdoor Attack Against Vision Language Modelscited by 35
- 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
- 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
- 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targetscited by 0
- 2026Backdooring Masked Diffusion Language Modelscited by 0
- 2026Backdooring Masked Diffusion Language Models19 May 2026
- 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targets10 May 2026
Similar, not cited
Close in wording, no citation either way
- 2024Diff-Cleanse: Identifying and Mitigating Backdoor Attacks in Diffusion Models
- 2024PureDiffusion: Using Backdoor to Counter Backdoor in Generative Diffusion Models
- 2025A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models
- 2025Backdoors in Conditional Diffusion: Threats to Responsible Synthetic Data Pipelines
- 2026Awakening the Hydra: Stabilizing Multi-Concept Backdoor Injection in Text-to-Image Diffusion Models
- 2024Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models
- 2025Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
- 2025Data-Chain Backdoor: Do You Trust Diffusion Models as Generative Data Supplier?