mlsanityData to 14 Oct 2026

Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variation

arXiv 2503.06453 ↗First posted 9 Mar 2025cs.CRcited by 8

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

In recent years, text-to-image (T2I) diffusion models have gained significant attention for their ability to generate high quality images reflecting text prompts. However, their growing popularity has also led to the emergence of backdoor threats, posing substantial risks. Currently, effective defense strategies against such threats are lacking due to the diversity of backdoor targets in T2I synthesis. In this paper, we propose NaviT2I, an efficient input-level backdoor defense framework against diverse T2I backdoors. Our approach is based on the new observation that trigger tokens tend to induce significant neuron activation variation in the early stage of the diffusion generation process, a phenomenon we term Early-step Activation Variation. Leveraging this insight, NaviT2I navigates T2I models to prevent malicious inputs by analyzing Neuron activation variations caused by input tokens. Extensive experiments show that NaviT2I significantly outperforms the baselines in both effectiveness and efficiency across diverse datasets, various T2I backdoors, and different model architectures including UNet and DiT. Furthermore, we show that our method remains effective under potential adaptive attacks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 59 of its 81 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

59 of 81 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models26 Feb
  2. 2024TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors9 Sep
  3. 2024Diff-Cleanse: Identifying and Mitigating Backdoor Attacks in Diffusion Models31 Jul
  4. 2024T2IShield: Defending Against Backdoors on Text-to-Image Diffusion Models5 Jul
  5. 2024Your Diffusion Model is Secretly a Certifiably Robust Classifier4 Feb
  6. 2023Backdooring Textual Inversion for Concept Censorship21 Aug
  7. 2023VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models12 Jun
  8. 2023Robust Classification via a Single Diffusion Model24 May
  9. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning7 May

Other references · 50

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  2. 2016Generative Adversarial Text to Image Synthesiscited by 735
  3. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  4. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  5. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  6. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  7. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
  8. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  9. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 235
  10. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 222
  11. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
  12. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 183
  13. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 158
  14. 2015Generating Images from Captions with Attentioncited by 136
  15. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
  16. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 104
  17. 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 97
  18. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 86
  19. 2020On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shapingcited by 83
  20. 2022Data-free Backdoor Removal based on Channel Lipschitznesscited by 79
  21. 2020DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentationcited by 77
  22. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
  23. 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 73
  24. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 71
  25. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 58
  26. 2023Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistencycited by 46
  27. 2022BadDet: Backdoor Attacks on Object Detectioncited by 45
  28. 2024IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistencycited by 39
  29. 2023An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesiscited by 31
  30. 2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
  31. 2022Kallima: A Clean-label Framework for Textual Backdoor Attackscited by 24
  32. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 21
  33. 2023UMD: Unsupervised Model Detection for X2X Backdoor Attackscited by 18
  34. 2021The Devil is in the GAN: Backdoor Attacks and Defenses in Deep Generative Modelscited by 12
  35. 2021Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterioncited by 11
  36. 2024Text-to-Image Synthesis: A Decade Surveycited by 9
  37. 2023NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learningcited by 8
  38. 2025Towards the Worst-case Robustness of Large Language Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviationcited by 1
  2. 2026AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasingcited by 1
  3. 2026What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamicscited by 1
  4. 2026Semantic-level Backdoor Attack against Text-to-Image Diffusion Modelscited by 0
  5. 2026Scaling Exposes the Trigger: Input-Level Backdoor Detection in Text-to-Image Diffusion Models via Cross-Attention Scalingcited by 0
  6. 2026Beyond the False Trade-off: Adaptive EWC for Stealthy and Generalizable T2I Backdoorscited by 0
  7. 2026Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoorcited by 0
  8. 2026Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetrycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models
  2. 2025Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models
  3. 2026Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Models
  4. 2026Awakening the Hydra: Stabilizing Multi-Concept Backdoor Injection in Text-to-Image Diffusion Models
  5. 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis
  6. 2024Defending Text-to-image Diffusion Models: Surprising Efficacy of Textual Perturbations Against Backdoor Attacks
  7. 2025PEPPER: Perception-Guided Perturbation for Robust Backdoor Defense in Text-to-Image Diffusion Models
  8. 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models